Jul 28 17:22:57.903213 ip-10-0-141-165 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 17:22:58.323457 ip-10-0-141-165 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:22:58.323457 ip-10-0-141-165 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 17:22:58.323457 ip-10-0-141-165 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:22:58.323457 ip-10-0-141-165 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 17:22:58.323457 ip-10-0-141-165 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:22:58.325877 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.325695    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 17:22:58.331298 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331278    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:22:58.331298 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331297    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:22:58.331298 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331300    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331304    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331309    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331312    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331315    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331319    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331321    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331324    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331327    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331329    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331332    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331335    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331338    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331340    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331343    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331345    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331348    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331351    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331353    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331356    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:22:58.331400 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331358    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331361    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331364    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331368    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331373    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331376    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331379    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331382    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331385    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331388    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331391    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331394    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331397    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331400    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331403    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331406    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331409    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331412    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331415    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:22:58.331937 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331419    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331423    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331426    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331428    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331432    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331435    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331437    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331440    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331443    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331445    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331448    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331450    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331453    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331455    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331458    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331460    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331463    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331466    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331469    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331472    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:22:58.332399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331474    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331477    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331479    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331482    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331484    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331487    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331489    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331493    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331495    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331498    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331501    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331503    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331506    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331509    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331512    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331514    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331517    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331519    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331522    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331524    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:22:58.332907 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331527    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331531    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331535    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331537    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331541    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331544    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331547    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331549    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331552    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331555    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331558    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331560    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331563    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331565    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331568    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331570    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331573    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.331576    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332299    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332306    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:22:58.333384 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332309    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332312    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332314    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332317    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332320    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332323    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332326    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332328    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332333    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332337    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332340    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332343    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332346    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332349    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332352    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332354    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332357    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332360    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332364    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332366    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:22:58.333882 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332369    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332373    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332376    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332379    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332381    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332384    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332386    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332390    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332392    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332395    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332398    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332401    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332404    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332408    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332411    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332414    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332417    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332419    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332424    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:22:58.334419 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332427    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332430    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332433    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332436    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332438    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332441    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332443    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332446    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332448    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332451    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332453    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332456    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332459    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332461    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332464    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332467    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332469    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332472    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332474    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:22:58.334890 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332477    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332480    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332482    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332485    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332487    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332490    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332493    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332495    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332498    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332500    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332503    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332506    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332509    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332511    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332514    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332516    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332520    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332523    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332526    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332528    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:22:58.335353 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332531    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332533    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332536    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332538    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332541    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332543    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332545    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332548    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332551    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332553    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332556    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332558    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332561    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332564    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332567    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332569    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332572    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332574    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332576    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332977    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:22:58.335844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332983    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332986    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332989    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332992    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332995    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.332997    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333000    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333003    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333006    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333008    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333011    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333014    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333017    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333019    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333022    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333024    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333027    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333029    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333032    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333034    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:22:58.336366 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333037    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333040    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333042    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333045    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333048    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333050    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333053    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333056    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333058    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333061    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333063    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333066    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333068    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333071    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333073    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333076    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333079    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333082    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333084    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333087    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:22:58.336856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333090    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333094    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333098    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333101    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333104    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333107    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333110    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333112    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333115    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333118    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333120    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333126    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333129    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333132    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333135    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333137    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333140    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333142    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333145    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:22:58.337378 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333148    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333151    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333154    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333157    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333160    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333163    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333165    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333168    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333171    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333173    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333176    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333179    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333181    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333184    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333186    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333189    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333191    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333194    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333196    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:22:58.337844 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333198    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333202    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333205    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333208    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333211    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333215    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333219    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333221    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333224    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333227    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333229    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333232    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333235    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333237    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333239    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333242    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333245    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333247    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333250    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:22:58.338319 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333252    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333325    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333333    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333340    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333346    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333350    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333353    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333358    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333362    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333365    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333368    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333372    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333375    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333378    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333381    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333384    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333387    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333390    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333394    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333398    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333401    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333405    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333408    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333411    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 17:22:58.338783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333415    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333418    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333421    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333424    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333427    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333430    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333433    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333437    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333440    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333444    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333447    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333450    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333453    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333456    2556 flags.go:64] FLAG: --enable-server="true"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333460    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333465    2556 flags.go:64] FLAG: --event-burst="100"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333468    2556 flags.go:64] FLAG: --event-qps="50"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333471    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333474    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333476    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333480    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333483    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333486    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333489    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333492    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 17:22:58.339379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333495    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333498    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333501    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333504    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333507    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333510    2556 flags.go:64] FLAG: --feature-gates=""
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333514    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333517    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333520    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333523    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333526    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333529    2556 flags.go:64] FLAG: --help="false"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333532    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333535    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333538    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333541    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333545    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333548    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333551    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333554    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333557    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333560    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333563    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333566    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 17:22:58.340046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333569    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333572    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333575    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333578    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333580    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333583    2556 flags.go:64] FLAG: --lock-file=""
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333586    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333588    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333591    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333597    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333601    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333604    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333607    2556 flags.go:64] FLAG: --logging-format="text"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333610    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333614    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333616    2556 flags.go:64] FLAG: --manifest-url=""
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333619    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333623    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333626    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333630    2556 flags.go:64] FLAG: --max-pods="110"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333633    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333636    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333639    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333642    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333645    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 17:22:58.340667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333648    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333650    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333657    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333661    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333664    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333667    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333670    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333676    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333679    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333682    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333685    2556 flags.go:64] FLAG: --port="10250"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333688    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333690    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00049e07cfcd0d0cf"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333694    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333697    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333700    2556 flags.go:64] FLAG: --register-node="true"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333702    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333706    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333710    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333713    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333716    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333720    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333723    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333728    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333731    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 17:22:58.341297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333734    2556 flags.go:64] FLAG: --runonce="false"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333737    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333740    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333743    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333746    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333749    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333752    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333755    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333758    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333761    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333764    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333767    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333769    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333772    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333776    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333779    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333784    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333787    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333789    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333793    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333796    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333799    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333802    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333805    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333807    2556 flags.go:64] FLAG: --v="2"
Jul 28 17:22:58.342033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333812    2556 flags.go:64] FLAG: --version="false"
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333816    2556 flags.go:64] FLAG: --vmodule=""
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333820    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.333824    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333952    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333958    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333961    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333963    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333967    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333970    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333973    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333975    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333978    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333980    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333984    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333988    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333991    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333994    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.333997    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334000    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:22:58.342649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334003    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334006    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334009    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334012    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334015    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334018    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334021    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334023    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334026    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334028    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334031    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334033    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334037    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334040    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334043    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334045    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334048    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334052    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334055    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334057    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:22:58.343155 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334060    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334064    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334068    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334071    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334074    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334076    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334079    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334081    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334084    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334087    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334089    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334092    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334094    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334097    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334100    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334103    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334105    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334108    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334110    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:22:58.343701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334113    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334116    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334118    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334121    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334123    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334129    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334132    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334134    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334137    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334139    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334143    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334146    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334148    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334151    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334153    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334156    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334158    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334161    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334164    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334166    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:22:58.344173 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334169    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334171    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334174    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334178    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334181    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334184    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334187    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334190    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334192    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334196    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334198    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334201    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334203    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334206    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334209    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334211    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334214    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334217    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334220    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334223    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:22:58.344661 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334225    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334228    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334232    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.334234    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.334825    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.341752    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.341769    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.341788    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.341795    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341854    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341860    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341863    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341881    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341884    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341887    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:22:58.345209 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341890    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341893    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341896    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341899    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341901    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341905    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341909    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341911    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341914    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341917    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341920    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341922    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341925    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341928    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341930    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341933    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341935    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341938    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341940    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341943    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:22:58.345594 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341946    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341949    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341951    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341956    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341959    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341962    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341964    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341967    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341969    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341972    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341974    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341977    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341979    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341982    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341984    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341987    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341990    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341992    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341995    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.341997    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:22:58.346107 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342000    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342003    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342005    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342008    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342010    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342014    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342018    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342022    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342025    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342028    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342031    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342034    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342037    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342040    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342043    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342046    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342050    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342053    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342055    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:22:58.346595 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342058    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342061    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342063    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342065    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342068    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342070    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342073    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342075    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342079    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342081    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342084    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342087    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342090    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342092    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342095    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342098    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342100    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342103    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342106    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342108    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:22:58.347122 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342111    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342114    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342117    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342119    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342121    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342124    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342126    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342129    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342132    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342135    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342138    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342140    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342143    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342145    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.342151    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:22:58.347619 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342264    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342269    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342272    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342275    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342277    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342280    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342282    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342285    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342287    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342290    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342293    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342295    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342298    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342300    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342303    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342306    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342308    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342311    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342313    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342316    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:22:58.348008 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342318    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342321    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342323    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342326    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342329    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342332    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342334    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342337    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342341    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342345    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342348    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342351    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342353    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342356    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342359    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342361    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342364    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342366    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342369    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342371    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:22:58.348487 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342374    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342376    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342379    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342382    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342384    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342387    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342389    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342392    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342395    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342397    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342400    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342403    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342405    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342408    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342410    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342413    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342416    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342418    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342421    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342424    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:22:58.349058 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342426    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342429    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342432    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342436    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342439    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342442    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342444    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342448    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342451    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342454    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342456    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342459    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342462    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342465    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342467    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342470    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342473    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342477    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342479    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:22:58.349540 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342482    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342484    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342487    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342489    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342492    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342494    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342497    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342499    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342502    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342505    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342507    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342510    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342512    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342515    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342517    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342520    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342522    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342525    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342527    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:22:58.350047 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:58.342530    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:22:58.350505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.342534    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:22:58.350505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.342657    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 17:22:58.350505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.345098    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 17:22:58.350505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.346653    2556 server.go:1019] "Starting client certificate rotation"
Jul 28 17:22:58.350505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.346748    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:22:58.350505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.346777    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:22:58.350505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.349963    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:22:58.354486 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.354462    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:22:58.365481 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.365459    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 28 17:22:58.367918 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.367902    2556 log.go:25] "Validated CRI v1 image API"
Jul 28 17:22:58.368749 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.368727    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 17:22:58.371974 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.371949    2556 fs.go:135] Filesystem UUIDs: map[57f36570-7b49-41c0-9af9-c4eedb43af7b:/dev/nvme0n1p4 66dcaf14-01c4-40e5-84f9-9011550ff017:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 28 17:22:58.372040 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.371972    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 17:22:58.377127 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.377016    2556 manager.go:216] Machine: {Timestamp:2026-07-28 17:22:58.375736873 +0000 UTC m=+0.321476029 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3083739 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b3c0729a73466e4849a34a04e8de8 SystemUUID:ec2b3c07-29a7-3466-e484-9a34a04e8de8 BootID:aab00887-2dda-4944-9f71-8d87a533b1da Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:bf:8a:13:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:bf:8a:13:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:d0:eb:8d:db:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 17:22:58.377127 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.377123    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 17:22:58.377248 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.377209    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 17:22:58.378789 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.378774    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:22:58.380375 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.380345    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 17:22:58.380521 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.380378    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 17:22:58.381375 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.381365    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 17:22:58.381420 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.381378    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 17:22:58.381420 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.381386    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:22:58.382110 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.382098    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:22:58.383350 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.383339    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 17:22:58.385307 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385294    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:22:58.385429 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385420    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 17:22:58.385462 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385451    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 17:22:58.385462 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385461    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 17:22:58.385513 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385480    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 17:22:58.385513 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385490    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 28 17:22:58.385513 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385500    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 17:22:58.385893 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385866    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:22:58.385943 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.385904    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:22:58.388243 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.388218    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 17:22:58.391443 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.391427    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 17:22:58.391506 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.391458    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 17:22:58.393473 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393460    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 17:22:58.393539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393481    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 17:22:58.393539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393490    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 17:22:58.393539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393498    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 17:22:58.393539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393511    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 17:22:58.393539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393519    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 17:22:58.393539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393530    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 17:22:58.393539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393537    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 17:22:58.393832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393548    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 17:22:58.393832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393557    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 17:22:58.393832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393569    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 17:22:58.393832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.393577    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 17:22:58.394361 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.394348    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 17:22:58.394361 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.394362    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 17:22:58.395831 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.395811    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 17:22:58.395923 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.395823    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 17:22:58.395973 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.395936    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 17:22:58.397789 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.397776    2556 server.go:1268] "Started kubelet"
Jul 28 17:22:58.397864 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.397842    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 17:22:58.398059 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.398013    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 17:22:58.398134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.398083    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 17:22:58.398619 ip-10-0-141-165 systemd[1]: Started Kubernetes Kubelet.
Jul 28 17:22:58.402306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.402280    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 17:22:58.403266 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.403228    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-92w4g"
Jul 28 17:22:58.403343 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.403330    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 17:22:58.404682 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.404664    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 17:22:58.411282 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.411264    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 17:22:58.411370 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.411283    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 17:22:58.411941 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.411654    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:58.411941 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.411844    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-92w4g"
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412113    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412133    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412267    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412316    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412323    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412336    2556 factory.go:55] Registering systemd factory
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412325    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412346    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412557    2556 factory.go:153] Registering CRI-O factory
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412569    2556 factory.go:223] Registration of the crio container factory successfully
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412592    2556 factory.go:103] Registering Raw factory
Jul 28 17:22:58.412926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.412615    2556 manager.go:1196] Started watching for new ooms in manager
Jul 28 17:22:58.413647 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.413024    2556 manager.go:349] Starting recovery of all containers
Jul 28 17:22:58.414796 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.414775    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 17:22:58.414902 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.414864    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 17:22:58.415987 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.414972    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-165.ec2.internal.18c683f229550df5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-165.ec2.internal,UID:ip-10-0-141-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-165.ec2.internal,},FirstTimestamp:2026-07-28 17:22:58.397744629 +0000 UTC m=+0.343483784,LastTimestamp:2026-07-28 17:22:58.397744629 +0000 UTC m=+0.343483784,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-165.ec2.internal,}"
Jul 28 17:22:58.420742 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.420699    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 17:22:58.423551 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.423480    2556 manager.go:354] Recovery completed
Jul 28 17:22:58.428286 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.428273    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:22:58.429988 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.429975    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:22:58.430036 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.430002    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:22:58.430036 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.430014    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:22:58.430516 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.430504    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 17:22:58.430576 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.430515    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 17:22:58.430576 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.430534    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:22:58.430576 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.430538    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:22:58.431175 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.431160    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:22:58.431245 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.431189    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:22:58.431245 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.431203    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:22:58.432244 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.432231    2556 policy_none.go:49] "None policy: Start"
Jul 28 17:22:58.432303 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.432247    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 17:22:58.432303 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.432257    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 17:22:58.433276 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.433265    2556 policy_none.go:47] "Start"
Jul 28 17:22:58.475809 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.475625    2556 manager.go:337] "Starting Device Plugin manager"
Jul 28 17:22:58.475956 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.475848    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 17:22:58.475956 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.475863    2556 server.go:87] "Starting device plugin registration server"
Jul 28 17:22:58.476162 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.476149    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 17:22:58.476208 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.476165    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 17:22:58.476289 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.476268    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 17:22:58.476383 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.476370    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 17:22:58.476455 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.476384    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 17:22:58.476981 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.476961    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 17:22:58.477062 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.477010    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:58.577302 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.577225    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:22:58.578700 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.578365    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:22:58.578700 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.578400    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:22:58.578700 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.578415    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:22:58.578700 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.578443    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.583963 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.583933    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 17:22:58.584033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.583999    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 17:22:58.584033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.584025    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 17:22:58.584135 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.584077    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 17:22:58.585027 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.585009    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.585117 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.585032    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-165.ec2.internal\": node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:58.586293 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.586279    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:22:58.638599 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.638566    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:58.686097 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.686055    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"]
Jul 28 17:22:58.686198 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.686179    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:22:58.688192 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.688169    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:22:58.688307 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.688204    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:22:58.688307 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.688215    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:22:58.689516 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.689503    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:22:58.689654 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.689636    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.689722 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.689671    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:22:58.690287 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.690270    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:22:58.690390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.690295    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:22:58.690390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.690305    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:22:58.690390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.690277    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:22:58.690390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.690367    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:22:58.690390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.690378    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:22:58.691489 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.691471    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.691583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.691498    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:22:58.692221 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.692202    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:22:58.692319 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.692233    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:22:58.692319 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.692245    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:22:58.715160 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.715127    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8e2da8fe16979baa8c3430923bef32a9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-165.ec2.internal\" (UID: \"8e2da8fe16979baa8c3430923bef32a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.715160 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.715163    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28bf49a37019b81f666ae5726a0f0c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"28bf49a37019b81f666ae5726a0f0c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.715363 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.715188    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28bf49a37019b81f666ae5726a0f0c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"28bf49a37019b81f666ae5726a0f0c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.720074 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.720053    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-165.ec2.internal\" not found" node="ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.724799 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.724781    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-165.ec2.internal\" not found" node="ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.738730 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.738710    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:58.815740 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.815701    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8e2da8fe16979baa8c3430923bef32a9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-165.ec2.internal\" (UID: \"8e2da8fe16979baa8c3430923bef32a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.815938 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.815745    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28bf49a37019b81f666ae5726a0f0c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"28bf49a37019b81f666ae5726a0f0c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.815938 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.815771    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28bf49a37019b81f666ae5726a0f0c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"28bf49a37019b81f666ae5726a0f0c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.815938 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.815824    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28bf49a37019b81f666ae5726a0f0c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"28bf49a37019b81f666ae5726a0f0c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.815938 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.815828    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8e2da8fe16979baa8c3430923bef32a9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-165.ec2.internal\" (UID: \"8e2da8fe16979baa8c3430923bef32a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.815938 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:58.815828    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28bf49a37019b81f666ae5726a0f0c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal\" (UID: \"28bf49a37019b81f666ae5726a0f0c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:58.839060 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.838997    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:58.940934 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:58.940886    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.023538 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.023504    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:59.030111 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.030093    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:59.041682 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.041657    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.143748 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.143652    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.243785 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.243746    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.344695 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.344655    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.345835 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.345816    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 17:22:59.345993 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.345975    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 17:22:59.418842 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.418731    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:17:58 +0000 UTC" deadline="2028-01-01 19:54:29.049927078 +0000 UTC"
Jul 28 17:22:59.418842 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.418777    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12530h31m29.63115285s"
Jul 28 17:22:59.445417 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.445389    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.464979 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.464949    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:22:59.525524 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.525493    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:22:59.545750 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.545712    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.608504 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:59.608463    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28bf49a37019b81f666ae5726a0f0c37.slice/crio-a5ae8b417b296cc63faaf3e014871567a7870c421d625a7d8167d18c335cd6b1 WatchSource:0}: Error finding container a5ae8b417b296cc63faaf3e014871567a7870c421d625a7d8167d18c335cd6b1: Status 404 returned error can't find the container with id a5ae8b417b296cc63faaf3e014871567a7870c421d625a7d8167d18c335cd6b1
Jul 28 17:22:59.609104 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:22:59.609075    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2da8fe16979baa8c3430923bef32a9.slice/crio-d4068d82428f09d1c4861aaa6ef9b886bc5f6b3fc766e43f603ddba440603f2b WatchSource:0}: Error finding container d4068d82428f09d1c4861aaa6ef9b886bc5f6b3fc766e43f603ddba440603f2b: Status 404 returned error can't find the container with id d4068d82428f09d1c4861aaa6ef9b886bc5f6b3fc766e43f603ddba440603f2b
Jul 28 17:22:59.614406 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.614375    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:22:59.646546 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.646504    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.748159 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.748105    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.848313 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:22:59.848284    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-165.ec2.internal\" not found"
Jul 28 17:22:59.849531 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.849509    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:22:59.912186 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.912146    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:59.928491 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.928466    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:22:59.929642 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.929623    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal"
Jul 28 17:22:59.939648 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:22:59.939626    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:23:00.389103 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.389068    2556 apiserver.go:52] "Watching apiserver"
Jul 28 17:23:00.398371 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.398347    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 17:23:00.400946 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.400919    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s","openshift-image-registry/node-ca-pq4hj","openshift-multus/multus-additional-cni-plugins-th8j5","openshift-multus/network-metrics-daemon-nlpg9","openshift-network-diagnostics/network-check-target-6gtv8","openshift-network-operator/iptables-alerter-qldwg","openshift-ovn-kubernetes/ovnkube-node-nrdnt","kube-system/global-pull-secret-syncer-t7pdd","kube-system/konnectivity-agent-mt4rt","kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gmzpv","openshift-dns/node-resolver-vpgqx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal","openshift-multus/multus-pt279"]
Jul 28 17:23:00.403699 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.403457    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.406188 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.406164    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.406585 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.406564    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.406754 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.406727    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 17:23:00.407097 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.407076    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-52j9s\""
Jul 28 17:23:00.408228 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.408082    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 17:23:00.408228 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.408221    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 17:23:00.408378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.408258    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 17:23:00.408378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.408074    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 17:23:00.408622 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.408597    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:23:00.408719 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.408701    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 17:23:00.408925 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.408908    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 17:23:00.409382 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.409360    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-55rpk\""
Jul 28 17:23:00.409587 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.409571    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 17:23:00.409661 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.409607    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hn7jw\""
Jul 28 17:23:00.409760 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.409738    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 17:23:00.409760 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.409754    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 17:23:00.409877 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.409771    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 17:23:00.409923 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.409912    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 17:23:00.411493 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.411472    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:00.411580 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.411554    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:00.412501 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.412480    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:00.412578 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.412548    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:00.412635 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.412591    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 17:23:00.414391 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.414262    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.415507 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.415488    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.416927 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.416643    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:23:00.416927 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.416711    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 17:23:00.416927 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.416709    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rpt4q\""
Jul 28 17:23:00.416927 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.416870    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 17:23:00.417264 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.417248    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.417442 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.417424    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 17:23:00.418516 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.418344    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n7j44\""
Jul 28 17:23:00.418516 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.418478    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 17:23:00.419231 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.418759    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.420119 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.420101    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.420215 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.420178    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:00.420420 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.420389    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hldxt\""
Jul 28 17:23:00.420600 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.420578    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 17:23:00.421241 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.421223    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7x44p\""
Jul 28 17:23:00.421365 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.421342    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.421436 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.421385    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 17:23:00.421808 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.421790    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 17:23:00.422091 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.422073    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 17:23:00.423252 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.423233    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.424764 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.424582    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 17:23:00.424896 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.424797    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 17:23:00.425137 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.425117    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lgs4h\""
Jul 28 17:23:00.425228 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.425182    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 17:23:00.425798 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.425778    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 17:23:00.425977 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.425851    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cfbqm\""
Jul 28 17:23:00.426043 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.426014    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 17:23:00.428079 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428058    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysctl-conf\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.428180 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428093    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-host\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.428180 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428131    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-tuned\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.428180 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428156    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.428344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428184    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-ovn\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.428344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428208    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-cni-bin\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.428344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428232    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cnibin\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.428344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428258    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.428344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428284    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:00.428344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428309    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-kubelet\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428347    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428385    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428409    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-modprobe-d\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428437    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-sys\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-lib-modules\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428500    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aabaac62-511e-4fcd-a3b7-e32fd7342d97-tmp\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428523    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-slash\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-env-overrides\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.428652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428635    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysconfig\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428685    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-node-log\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428713    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-log-socket\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428738    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-ovnkube-config\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428776    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f20edee-328b-4193-87de-90ad2efc3bb7-ovn-node-metrics-cert\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tml7q\" (UniqueName: \"kubernetes.io/projected/7f20edee-328b-4193-87de-90ad2efc3bb7-kube-api-access-tml7q\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428825    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-systemd\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-run\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428874    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljt76\" (UniqueName: \"kubernetes.io/projected/aabaac62-511e-4fcd-a3b7-e32fd7342d97-kube-api-access-ljt76\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428901    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-var-lib-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428926    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-ovnkube-script-lib\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.428967    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-system-cni-dir\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429005    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-os-release\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429060    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r4x9\" (UniqueName: \"kubernetes.io/projected/5e969c14-a4ea-49f0-ae8b-db7353c035ce-kube-api-access-8r4x9\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.429115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r87jd\" (UniqueName: \"kubernetes.io/projected/6ae9655b-4180-4a82-9fd7-9eff9b69d437-kube-api-access-r87jd\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429123    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-run-netns\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429154    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-systemd\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429179    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-etc-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysctl-d\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429227    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-var-lib-kubelet\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429252    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429279    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-systemd-units\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429305    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-cni-netd\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429332    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.430028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.429359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-kubernetes\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.430537 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.430149    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:23:00.455679 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.455655    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w64ns"
Jul 28 17:23:00.469519 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.469436    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w64ns"
Jul 28 17:23:00.513353 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.513298    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 17:23:00.530019 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.529987    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.530174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530034    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-sys\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.530174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530063    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-lib-modules\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.530174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530086    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aabaac62-511e-4fcd-a3b7-e32fd7342d97-tmp\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.530174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530112    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.530174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-socket-dir-parent\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.530174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530157    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-sys\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.530174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530170    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-cni-bin\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530237    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530301    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-lib-modules\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530390    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-slash\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530425    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-env-overrides\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530451    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-conf-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530475    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-dbus\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530499    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-etc-selinux\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530512    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-slash\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530525    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysconfig\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.530580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530568    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-os-release\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530608    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-node-log\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530636    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-ovnkube-config\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530648    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530666    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tml7q\" (UniqueName: \"kubernetes.io/projected/7f20edee-328b-4193-87de-90ad2efc3bb7-kube-api-access-tml7q\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530692    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-systemd\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530720    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljt76\" (UniqueName: \"kubernetes.io/projected/aabaac62-511e-4fcd-a3b7-e32fd7342d97-kube-api-access-ljt76\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530743    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/529a053a-5210-4070-929f-ac5325c9d04b-host\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530772    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrrxd\" (UniqueName: \"kubernetes.io/projected/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-kube-api-access-hrrxd\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530801    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-system-cni-dir\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530829    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r4x9\" (UniqueName: \"kubernetes.io/projected/5e969c14-a4ea-49f0-ae8b-db7353c035ce-kube-api-access-8r4x9\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530948    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r87jd\" (UniqueName: \"kubernetes.io/projected/6ae9655b-4180-4a82-9fd7-9eff9b69d437-kube-api-access-r87jd\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530976    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531006    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50c96cb9-c926-46eb-8783-993ffbeb9e43-tmp-dir\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531050    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-system-cni-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-etc-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.531189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531081    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-env-overrides\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531102    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-etc-kubernetes\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.530574    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysconfig\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531402    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-systemd\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531600    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-system-cni-dir\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531638    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-etc-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531658    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531818    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jx7n\" (UniqueName: \"kubernetes.io/projected/529a053a-5210-4070-929f-ac5325c9d04b-kube-api-access-6jx7n\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531838    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-node-log\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-socket-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531890    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-sys-fs\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531922    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-systemd-units\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.531971    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532013    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-ovnkube-config\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532030    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532030    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-kubernetes\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532084    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-kubernetes\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-systemd-units\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532155    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysctl-conf\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532183    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-host\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532208    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-tuned\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532234    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-cnibin\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532263    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-host\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532262    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-kubelet-config\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532301    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysctl-conf\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532323    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c544a44d-9a4a-4db5-956f-94711c0a23e3-agent-certs\") pod \"konnectivity-agent-mt4rt\" (UID: \"c544a44d-9a4a-4db5-956f-94711c0a23e3\") " pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532355    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/318d751f-c627-48f7-aa32-2f401c7e5606-multus-daemon-config\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532387    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532411    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-device-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532436    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-kubelet\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532462    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532486    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-modprobe-d\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532500    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-kubelet\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.532881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532514    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbqlv\" (UniqueName: \"kubernetes.io/projected/50c96cb9-c926-46eb-8783-993ffbeb9e43-kube-api-access-qbqlv\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532539    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw2n9\" (UniqueName: \"kubernetes.io/projected/2d1a5dac-488b-405b-9045-04bd32406da9-kube-api-access-bw2n9\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/318d751f-c627-48f7-aa32-2f401c7e5606-cni-binary-copy\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532585    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-kubelet\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532601    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/529a053a-5210-4070-929f-ac5325c9d04b-serviceca\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532616    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-registration-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532635    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50c96cb9-c926-46eb-8783-993ffbeb9e43-hosts-file\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532627    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-modprobe-d\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532669    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-cni-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532691    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-multus-certs\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532717    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-log-socket\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532745    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f20edee-328b-4193-87de-90ad2efc3bb7-ovn-node-metrics-cert\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532763    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-log-socket\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532773    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-run\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532801    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-os-release\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532822    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-run\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.533768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532830    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-k8s-cni-cncf-io\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532871    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-var-lib-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532902    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-os-release\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532913    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-ovnkube-script-lib\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d1a5dac-488b-405b-9045-04bd32406da9-iptables-alerter-script\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532971    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c544a44d-9a4a-4db5-956f-94711c0a23e3-konnectivity-ca\") pod \"konnectivity-agent-mt4rt\" (UID: \"c544a44d-9a4a-4db5-956f-94711c0a23e3\") " pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532995    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-netns\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533020    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-cni-multus\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533045    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg7g4\" (UniqueName: \"kubernetes.io/projected/318d751f-c627-48f7-aa32-2f401c7e5606-kube-api-access-kg7g4\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533070    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533106    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-run-netns\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-systemd\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533161    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysctl-d\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.532946    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-var-lib-openvswitch\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-var-lib-kubelet\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533233    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-cni-netd\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533246    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-run-netns\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.534568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533263    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533306    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533337    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-systemd\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533341    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d1a5dac-488b-405b-9045-04bd32406da9-host-slash\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533368    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-hostroot\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533396    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-ovn\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533422    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-cni-bin\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533437    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f20edee-328b-4193-87de-90ad2efc3bb7-ovnkube-script-lib\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533450    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cnibin\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533453    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-sysctl-d\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533479    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.533560    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533567    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-cni-netd\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.533644    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:01.033604014 +0000 UTC m=+2.979343182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533682    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aabaac62-511e-4fcd-a3b7-e32fd7342d97-var-lib-kubelet\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533688    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-run-ovn\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533726    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f20edee-328b-4193-87de-90ad2efc3bb7-host-cni-bin\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.535214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533914    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5e969c14-a4ea-49f0-ae8b-db7353c035ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.535920 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533978    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.535920 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.533982    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e969c14-a4ea-49f0-ae8b-db7353c035ce-cnibin\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.535920 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.535277    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aabaac62-511e-4fcd-a3b7-e32fd7342d97-tmp\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.538703 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.538676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aabaac62-511e-4fcd-a3b7-e32fd7342d97-etc-tuned\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.540914 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.540889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f20edee-328b-4193-87de-90ad2efc3bb7-ovn-node-metrics-cert\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.542112 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.542092    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:23:00.542204 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.542120    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:23:00.542204 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.542135    2556 projected.go:196] Error preparing data for projected volume kube-api-access-nww5l for pod openshift-network-diagnostics/network-check-target-6gtv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:00.542333 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.542209    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l podName:962ce334-76b4-49da-9402-ba293e765556 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:01.042189587 +0000 UTC m=+2.987928730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nww5l" (UniqueName: "kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l") pod "network-check-target-6gtv8" (UID: "962ce334-76b4-49da-9402-ba293e765556") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:00.543518 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.543439    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljt76\" (UniqueName: \"kubernetes.io/projected/aabaac62-511e-4fcd-a3b7-e32fd7342d97-kube-api-access-ljt76\") pod \"tuned-gmzpv\" (UID: \"aabaac62-511e-4fcd-a3b7-e32fd7342d97\") " pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.544340 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.543813    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r87jd\" (UniqueName: \"kubernetes.io/projected/6ae9655b-4180-4a82-9fd7-9eff9b69d437-kube-api-access-r87jd\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:00.544340 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.544287    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r4x9\" (UniqueName: \"kubernetes.io/projected/5e969c14-a4ea-49f0-ae8b-db7353c035ce-kube-api-access-8r4x9\") pod \"multus-additional-cni-plugins-th8j5\" (UID: \"5e969c14-a4ea-49f0-ae8b-db7353c035ce\") " pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.546738 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.546634    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tml7q\" (UniqueName: \"kubernetes.io/projected/7f20edee-328b-4193-87de-90ad2efc3bb7-kube-api-access-tml7q\") pod \"ovnkube-node-nrdnt\" (UID: \"7f20edee-328b-4193-87de-90ad2efc3bb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.588780 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.588723    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" event={"ID":"28bf49a37019b81f666ae5726a0f0c37","Type":"ContainerStarted","Data":"a5ae8b417b296cc63faaf3e014871567a7870c421d625a7d8167d18c335cd6b1"}
Jul 28 17:23:00.589849 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.589818    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal" event={"ID":"8e2da8fe16979baa8c3430923bef32a9","Type":"ContainerStarted","Data":"d4068d82428f09d1c4861aaa6ef9b886bc5f6b3fc766e43f603ddba440603f2b"}
Jul 28 17:23:00.633845 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.633803    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d1a5dac-488b-405b-9045-04bd32406da9-iptables-alerter-script\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.633857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c544a44d-9a4a-4db5-956f-94711c0a23e3-konnectivity-ca\") pod \"konnectivity-agent-mt4rt\" (UID: \"c544a44d-9a4a-4db5-956f-94711c0a23e3\") " pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.633884    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-netns\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.633908    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-cni-multus\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.633932    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg7g4\" (UniqueName: \"kubernetes.io/projected/318d751f-c627-48f7-aa32-2f401c7e5606-kube-api-access-kg7g4\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.633975    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634001    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-netns\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634018    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d1a5dac-488b-405b-9045-04bd32406da9-host-slash\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634027    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-cni-multus\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-hostroot\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634092    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-hostroot\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634101    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-socket-dir-parent\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.634113    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634130    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-cni-bin\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634137    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d1a5dac-488b-405b-9045-04bd32406da9-host-slash\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634161    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-conf-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:00.634192    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret podName:a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:01.134170424 +0000 UTC m=+3.079909573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret") pod "global-pull-secret-syncer-t7pdd" (UID: "a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634201    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-conf-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634222    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-cni-bin\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634225    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-dbus\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634191    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-socket-dir-parent\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634255    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-etc-selinux\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634284    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-os-release\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634316    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/529a053a-5210-4070-929f-ac5325c9d04b-host\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634342    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrrxd\" (UniqueName: \"kubernetes.io/projected/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-kube-api-access-hrrxd\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634384    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50c96cb9-c926-46eb-8783-993ffbeb9e43-tmp-dir\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634405    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-system-cni-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.634515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634404    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-dbus\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634429    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-os-release\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634503    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-etc-selinux\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634514    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-system-cni-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634543    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d1a5dac-488b-405b-9045-04bd32406da9-iptables-alerter-script\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634560    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/529a053a-5210-4070-929f-ac5325c9d04b-host\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-etc-kubernetes\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634588    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c544a44d-9a4a-4db5-956f-94711c0a23e3-konnectivity-ca\") pod \"konnectivity-agent-mt4rt\" (UID: \"c544a44d-9a4a-4db5-956f-94711c0a23e3\") " pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634597    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-etc-kubernetes\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634602    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jx7n\" (UniqueName: \"kubernetes.io/projected/529a053a-5210-4070-929f-ac5325c9d04b-kube-api-access-6jx7n\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-socket-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634658    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-sys-fs\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634680    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50c96cb9-c926-46eb-8783-993ffbeb9e43-tmp-dir\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634689    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-cnibin\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634706    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-kubelet-config\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634723    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c544a44d-9a4a-4db5-956f-94711c0a23e3-agent-certs\") pod \"konnectivity-agent-mt4rt\" (UID: \"c544a44d-9a4a-4db5-956f-94711c0a23e3\") " pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634735    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-sys-fs\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634740    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-cnibin\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.635399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634740    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/318d751f-c627-48f7-aa32-2f401c7e5606-multus-daemon-config\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634788    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-kubelet-config\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634809    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-device-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634840    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634841    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbqlv\" (UniqueName: \"kubernetes.io/projected/50c96cb9-c926-46eb-8783-993ffbeb9e43-kube-api-access-qbqlv\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634877    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw2n9\" (UniqueName: \"kubernetes.io/projected/2d1a5dac-488b-405b-9045-04bd32406da9-kube-api-access-bw2n9\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634901    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-device-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634907    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/318d751f-c627-48f7-aa32-2f401c7e5606-cni-binary-copy\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634939    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-kubelet\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634966    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/529a053a-5210-4070-929f-ac5325c9d04b-serviceca\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.634992    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-registration-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635017    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50c96cb9-c926-46eb-8783-993ffbeb9e43-hosts-file\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635023    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-var-lib-kubelet\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-cni-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635092    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-multus-cni-dir\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50c96cb9-c926-46eb-8783-993ffbeb9e43-hosts-file\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.636141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635145    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-multus-certs\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635171    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-registration-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635185    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-k8s-cni-cncf-io\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635208    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-multus-certs\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635252    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/318d751f-c627-48f7-aa32-2f401c7e5606-host-run-k8s-cni-cncf-io\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635320    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/318d751f-c627-48f7-aa32-2f401c7e5606-multus-daemon-config\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635481    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/529a053a-5210-4070-929f-ac5325c9d04b-serviceca\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635549    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-socket-dir\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.636766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.635738    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/318d751f-c627-48f7-aa32-2f401c7e5606-cni-binary-copy\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.637461 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.637443    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c544a44d-9a4a-4db5-956f-94711c0a23e3-agent-certs\") pod \"konnectivity-agent-mt4rt\" (UID: \"c544a44d-9a4a-4db5-956f-94711c0a23e3\") " pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.646321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.646253    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jx7n\" (UniqueName: \"kubernetes.io/projected/529a053a-5210-4070-929f-ac5325c9d04b-kube-api-access-6jx7n\") pod \"node-ca-pq4hj\" (UID: \"529a053a-5210-4070-929f-ac5325c9d04b\") " pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.646519 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.646497    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbqlv\" (UniqueName: \"kubernetes.io/projected/50c96cb9-c926-46eb-8783-993ffbeb9e43-kube-api-access-qbqlv\") pod \"node-resolver-vpgqx\" (UID: \"50c96cb9-c926-46eb-8783-993ffbeb9e43\") " pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:00.647116 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.647092    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw2n9\" (UniqueName: \"kubernetes.io/projected/2d1a5dac-488b-405b-9045-04bd32406da9-kube-api-access-bw2n9\") pod \"iptables-alerter-qldwg\" (UID: \"2d1a5dac-488b-405b-9045-04bd32406da9\") " pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.647479 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.647462    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrrxd\" (UniqueName: \"kubernetes.io/projected/c4fcdfe7-9b79-4cca-b06c-36c56ba51af8-kube-api-access-hrrxd\") pod \"aws-ebs-csi-driver-node-bjv2s\" (UID: \"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.648028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.648007    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg7g4\" (UniqueName: \"kubernetes.io/projected/318d751f-c627-48f7-aa32-2f401c7e5606-kube-api-access-kg7g4\") pod \"multus-pt279\" (UID: \"318d751f-c627-48f7-aa32-2f401c7e5606\") " pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.699190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.699145    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:23:00.725634 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.725597    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:00.733642 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.733608    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gmzpv"
Jul 28 17:23:00.744755 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.744728    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-th8j5"
Jul 28 17:23:00.754770 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.754742    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qldwg"
Jul 28 17:23:00.772847 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.772806    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:00.790293 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.790261    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pt279"
Jul 28 17:23:00.799244 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.799221    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pq4hj"
Jul 28 17:23:00.810011 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.809987    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s"
Jul 28 17:23:00.815886 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:00.815861    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vpgqx"
Jul 28 17:23:01.038932 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:01.038873    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:01.039148 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.039058    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:01.039148 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.039141    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:02.039119348 +0000 UTC m=+3.984858496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:01.140224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:01.140179    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:01.140411 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:01.140276    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:01.140411 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.140351    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:01.140510 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.140421    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret podName:a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:02.140405586 +0000 UTC m=+4.086144729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret") pod "global-pull-secret-syncer-t7pdd" (UID: "a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:01.140510 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.140441    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:23:01.140510 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.140461    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:23:01.140510 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.140475    2556 projected.go:196] Error preparing data for projected volume kube-api-access-nww5l for pod openshift-network-diagnostics/network-check-target-6gtv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:01.140676 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.140537    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l podName:962ce334-76b4-49da-9402-ba293e765556 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:02.140519538 +0000 UTC m=+4.086258688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nww5l" (UniqueName: "kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l") pod "network-check-target-6gtv8" (UID: "962ce334-76b4-49da-9402-ba293e765556") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:01.472798 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:01.472730    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:18:00 +0000 UTC" deadline="2028-04-30 18:53:39.312868556 +0000 UTC"
Jul 28 17:23:01.472798 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:01.472790    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15409h30m37.840083698s"
Jul 28 17:23:01.584761 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:01.584728    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:01.584922 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:01.584871    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:01.626302 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:01.626257    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4fcdfe7_9b79_4cca_b06c_36c56ba51af8.slice/crio-fa37568b1969233aa51c9fcfcaee92ff19b0d14b859a9bb88a4a979a3cae849c WatchSource:0}: Error finding container fa37568b1969233aa51c9fcfcaee92ff19b0d14b859a9bb88a4a979a3cae849c: Status 404 returned error can't find the container with id fa37568b1969233aa51c9fcfcaee92ff19b0d14b859a9bb88a4a979a3cae849c
Jul 28 17:23:01.627379 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:01.627353    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318d751f_c627_48f7_aa32_2f401c7e5606.slice/crio-66ce8556ea91bfffac10ac0af7121677f546922fc8d1a46e3a44ed01c91f65ea WatchSource:0}: Error finding container 66ce8556ea91bfffac10ac0af7121677f546922fc8d1a46e3a44ed01c91f65ea: Status 404 returned error can't find the container with id 66ce8556ea91bfffac10ac0af7121677f546922fc8d1a46e3a44ed01c91f65ea
Jul 28 17:23:01.628306 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:01.628278    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1a5dac_488b_405b_9045_04bd32406da9.slice/crio-94eb9493ead78e22f22ee1def255ed9451751a512996068f1e4885dc7448f64f WatchSource:0}: Error finding container 94eb9493ead78e22f22ee1def255ed9451751a512996068f1e4885dc7448f64f: Status 404 returned error can't find the container with id 94eb9493ead78e22f22ee1def255ed9451751a512996068f1e4885dc7448f64f
Jul 28 17:23:01.633072 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:01.632757    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c96cb9_c926_46eb_8783_993ffbeb9e43.slice/crio-2896944ce31287aa8f643103bf9a25a31b8f2df02a940c068f22c0aa640d7953 WatchSource:0}: Error finding container 2896944ce31287aa8f643103bf9a25a31b8f2df02a940c068f22c0aa640d7953: Status 404 returned error can't find the container with id 2896944ce31287aa8f643103bf9a25a31b8f2df02a940c068f22c0aa640d7953
Jul 28 17:23:01.634128 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:01.634105    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod529a053a_5210_4070_929f_ac5325c9d04b.slice/crio-69fead7ced71dfa10a81ba4605f63a0a3de5834b135569d88715f0340a82ae6a WatchSource:0}: Error finding container 69fead7ced71dfa10a81ba4605f63a0a3de5834b135569d88715f0340a82ae6a: Status 404 returned error can't find the container with id 69fead7ced71dfa10a81ba4605f63a0a3de5834b135569d88715f0340a82ae6a
Jul 28 17:23:02.048277 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.047988    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:02.048476 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.048182    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:02.048476 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.048365    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:04.048339836 +0000 UTC m=+5.994078982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:02.149088 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.148993    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:02.149088 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.149077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:02.149346 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.149202    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:02.149346 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.149269    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret podName:a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:04.149246704 +0000 UTC m=+6.094985852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret") pod "global-pull-secret-syncer-t7pdd" (UID: "a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:02.149722 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.149703    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:23:02.149799 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.149731    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:23:02.149799 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.149746    2556 projected.go:196] Error preparing data for projected volume kube-api-access-nww5l for pod openshift-network-diagnostics/network-check-target-6gtv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:02.149915 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.149837    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l podName:962ce334-76b4-49da-9402-ba293e765556 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:04.149820489 +0000 UTC m=+6.095559641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nww5l" (UniqueName: "kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l") pod "network-check-target-6gtv8" (UID: "962ce334-76b4-49da-9402-ba293e765556") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:02.473317 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.473239    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:18:00 +0000 UTC" deadline="2028-01-07 13:27:30.558515116 +0000 UTC"
Jul 28 17:23:02.473317 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.473305    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12668h4m28.085213989s"
Jul 28 17:23:02.588089 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.587322    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:02.588089 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.587466    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:02.588089 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.587867    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:02.588089 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:02.587983    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:02.604138 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.604052    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s" event={"ID":"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8","Type":"ContainerStarted","Data":"fa37568b1969233aa51c9fcfcaee92ff19b0d14b859a9bb88a4a979a3cae849c"}
Jul 28 17:23:02.618995 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.618549    2556 generic.go:356] "Generic (PLEG): container finished" podID="28bf49a37019b81f666ae5726a0f0c37" containerID="7bae2ee8e00c3a6a2ed2f1e2b624470c3c9c7703948d4bf9602fc066be27b9e0" exitCode=0
Jul 28 17:23:02.619169 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.618998    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" event={"ID":"28bf49a37019b81f666ae5726a0f0c37","Type":"ContainerDied","Data":"7bae2ee8e00c3a6a2ed2f1e2b624470c3c9c7703948d4bf9602fc066be27b9e0"}
Jul 28 17:23:02.629489 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.629375    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"0749332756b19704b9cbc445e59f643a2ff9c343080ccee71352269d10b2fa04"}
Jul 28 17:23:02.634335 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.634224    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerStarted","Data":"ba3d27cf0a09a1933a6c4737fbe7e50434ec2552e5a15ae2c1d108917b814245"}
Jul 28 17:23:02.657437 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.657371    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vpgqx" event={"ID":"50c96cb9-c926-46eb-8783-993ffbeb9e43","Type":"ContainerStarted","Data":"2896944ce31287aa8f643103bf9a25a31b8f2df02a940c068f22c0aa640d7953"}
Jul 28 17:23:02.682002 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.681938    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qldwg" event={"ID":"2d1a5dac-488b-405b-9045-04bd32406da9","Type":"ContainerStarted","Data":"94eb9493ead78e22f22ee1def255ed9451751a512996068f1e4885dc7448f64f"}
Jul 28 17:23:02.695658 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.694899    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal" event={"ID":"8e2da8fe16979baa8c3430923bef32a9","Type":"ContainerStarted","Data":"657f29aa67260955afc60d7cd3dfecb855b1849ec7b87ed048e54cfee9551628"}
Jul 28 17:23:02.707425 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.707391    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gmzpv" event={"ID":"aabaac62-511e-4fcd-a3b7-e32fd7342d97","Type":"ContainerStarted","Data":"16bf13fc5b127369b88edbf4194eca02659a13f5789f5841324139f3136afbd6"}
Jul 28 17:23:02.711917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.711877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pq4hj" event={"ID":"529a053a-5210-4070-929f-ac5325c9d04b","Type":"ContainerStarted","Data":"69fead7ced71dfa10a81ba4605f63a0a3de5834b135569d88715f0340a82ae6a"}
Jul 28 17:23:02.731061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.730968    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mt4rt" event={"ID":"c544a44d-9a4a-4db5-956f-94711c0a23e3","Type":"ContainerStarted","Data":"930a18c54c0aecdd9b5324ab69b6e8980437bc02bc2fe53dbac544baa7ddee8f"}
Jul 28 17:23:02.749915 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:02.749877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pt279" event={"ID":"318d751f-c627-48f7-aa32-2f401c7e5606","Type":"ContainerStarted","Data":"66ce8556ea91bfffac10ac0af7121677f546922fc8d1a46e3a44ed01c91f65ea"}
Jul 28 17:23:03.585272 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:03.585233    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:03.585784 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:03.585380    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:03.796924 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:03.796864    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" event={"ID":"28bf49a37019b81f666ae5726a0f0c37","Type":"ContainerStarted","Data":"8b66069c23e64d644c8d7e36daf16f2ecf7c5a97fabd3c7b0023197d934427fb"}
Jul 28 17:23:03.819739 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:03.819613    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-165.ec2.internal" podStartSLOduration=3.819587743 podStartE2EDuration="3.819587743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:23:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:02.71406876 +0000 UTC m=+4.659807929" watchObservedRunningTime="2026-07-28 17:23:03.819587743 +0000 UTC m=+5.765326911"
Jul 28 17:23:04.069621 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:04.069575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:04.069830 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.069786    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:04.069896 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.069854    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:08.069833834 +0000 UTC m=+10.015572983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:04.170316 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:04.170274    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:04.170508 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:04.170371    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:04.170599 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.170548    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:23:04.170599 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.170569    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:23:04.170599 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.170583    2556 projected.go:196] Error preparing data for projected volume kube-api-access-nww5l for pod openshift-network-diagnostics/network-check-target-6gtv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:04.170764 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.170649    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l podName:962ce334-76b4-49da-9402-ba293e765556 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:08.170628009 +0000 UTC m=+10.116367156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nww5l" (UniqueName: "kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l") pod "network-check-target-6gtv8" (UID: "962ce334-76b4-49da-9402-ba293e765556") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:04.170764 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.170735    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:04.170887 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.170772    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret podName:a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:08.170760339 +0000 UTC m=+10.116499486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret") pod "global-pull-secret-syncer-t7pdd" (UID: "a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:04.586418 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:04.586386    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:04.586949 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:04.586623    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:04.586949 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.586741    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:04.586949 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:04.586770    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:05.585393 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:05.585292    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:05.585585 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:05.585422    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:06.586497 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:06.585975    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:06.586497 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:06.586123    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:06.587446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:06.587164    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:06.587446 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:06.587283    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:07.584416 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:07.584379    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:07.584582 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:07.584518    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:08.102998 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:08.102929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:08.103490 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.103056    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:08.103490 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.103128    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:16.103107693 +0000 UTC m=+18.048846835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:08.204246 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:08.204206    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:08.204423 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:08.204329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:08.209630 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.204519    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:23:08.209630 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.204556    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:23:08.209630 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.204571    2556 projected.go:196] Error preparing data for projected volume kube-api-access-nww5l for pod openshift-network-diagnostics/network-check-target-6gtv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:08.209630 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.204645    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l podName:962ce334-76b4-49da-9402-ba293e765556 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:16.204624756 +0000 UTC m=+18.150363902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nww5l" (UniqueName: "kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l") pod "network-check-target-6gtv8" (UID: "962ce334-76b4-49da-9402-ba293e765556") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:08.209630 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.205138    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:08.209630 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.205199    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret podName:a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:16.205184482 +0000 UTC m=+18.150923627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret") pod "global-pull-secret-syncer-t7pdd" (UID: "a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:08.586278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:08.586018    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:08.586278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:08.586069    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:08.586278 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.586146    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:08.586608 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:08.586584    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:09.585467 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:09.584958    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:09.585467 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:09.585093    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:10.585175 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:10.585141    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:10.585342 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:10.585274    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:10.585421 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:10.585370    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:10.585521 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:10.585472    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:11.584995 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:11.584964    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:11.585200 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:11.585097    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:12.585306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:12.585215    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:12.585783 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:12.585364    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:12.585783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:12.585419    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:12.587366 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:12.586808    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:13.585030 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:13.584994    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:13.585183 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:13.585113    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:14.585247 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:14.585211    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:14.585700 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:14.585212    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:14.585700 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:14.585353    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:14.585700 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:14.585403    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:15.584855 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:15.584818    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:15.585084 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:15.584965    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:16.160889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:16.160842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:16.161306 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.160983    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:16.161306 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.161045    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:32.161029454 +0000 UTC m=+34.106768600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:16.261842 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:16.261811    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:16.262025 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:16.261863    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:16.262025 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.261975    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:16.262115 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.262037    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret podName:a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:32.262021076 +0000 UTC m=+34.207760218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret") pod "global-pull-secret-syncer-t7pdd" (UID: "a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:16.262115 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.261981    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:23:16.262115 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.262067    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:23:16.262115 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.262076    2556 projected.go:196] Error preparing data for projected volume kube-api-access-nww5l for pod openshift-network-diagnostics/network-check-target-6gtv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:16.262115 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.262115    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l podName:962ce334-76b4-49da-9402-ba293e765556 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:32.262104681 +0000 UTC m=+34.207843822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nww5l" (UniqueName: "kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l") pod "network-check-target-6gtv8" (UID: "962ce334-76b4-49da-9402-ba293e765556") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:16.584726 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:16.584692    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:16.584920 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:16.584736    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:16.584920 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.584836    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:16.585053 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:16.584983    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:17.584610 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:17.584575    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:17.585066 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:17.584701    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:18.586033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:18.585995    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:18.586033 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:18.586033    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:18.586497 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:18.586133    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:18.586497 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:18.586284    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:19.584986 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:19.584941    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:19.585170 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:19.585096    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:20.585302 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.584889    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:20.585302 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.584944    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:20.586231 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:20.585371    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:20.586231 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:20.585471    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:20.833364 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.833331    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gmzpv" event={"ID":"aabaac62-511e-4fcd-a3b7-e32fd7342d97","Type":"ContainerStarted","Data":"722e7975c8c3be30c38b1b4d6036fbed2b3fc45a76177d8d15ae743e4d8c8b1b"}
Jul 28 17:23:20.835068 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.835041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pq4hj" event={"ID":"529a053a-5210-4070-929f-ac5325c9d04b","Type":"ContainerStarted","Data":"d8327aee01fc3bb46a66ab40e5bf9bfe2c863a7eb6c3a14b30ff1b31e4e0b2ca"}
Jul 28 17:23:20.837233 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.837177    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mt4rt" event={"ID":"c544a44d-9a4a-4db5-956f-94711c0a23e3","Type":"ContainerStarted","Data":"cd635e10ac5439c5f640a50011097e972b7fd92707efacf7650833d8c755f832"}
Jul 28 17:23:20.839061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.839030    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pt279" event={"ID":"318d751f-c627-48f7-aa32-2f401c7e5606","Type":"ContainerStarted","Data":"e448d9919950b1a21af1ebde3bc713a1611bcdab7ec87b01cb191f26e0daf764"}
Jul 28 17:23:20.840786 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.840766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s" event={"ID":"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8","Type":"ContainerStarted","Data":"c5a401fe32f1552a47da7552584718df09886e685981e72c921c3cc0c9fe5c27"}
Jul 28 17:23:20.844860 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.844835    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"c79b6f6d3ca0727dddc81c26cbb50e3d083e880be47c734f7c0d5f7dd6372550"}
Jul 28 17:23:20.844979 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.844928    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"4dbed4eea248c4ad52008ce506f70c2a8d607dd9d4511067535831af4723d187"}
Jul 28 17:23:20.845045 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.845022    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"b45eee807eb7d0d7c5634fff3f868e1a97c1039f011710e53d0910f410445acf"}
Jul 28 17:23:20.845104 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.845075    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"0b2255e2549b1fe223f7293144f6c9a5cdea318c73b5f61e279b0d3d6413de84"}
Jul 28 17:23:20.845157 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.845123    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"0c8a01398508eb6138247a0bfdbb4e7666a03fc0941e7f8039adbeaca83cf033"}
Jul 28 17:23:20.845271 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.845255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"598394ec82cc1c4b7621c28219b1b9c722f783220020d692bde99d0363fe7d3f"}
Jul 28 17:23:20.846598 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.846576    2556 generic.go:356] "Generic (PLEG): container finished" podID="5e969c14-a4ea-49f0-ae8b-db7353c035ce" containerID="d45a3696a533480b4b31e8baaeb62e77add454764b15660f2ed6e36fd81c94ec" exitCode=0
Jul 28 17:23:20.846714 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.846643    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerDied","Data":"d45a3696a533480b4b31e8baaeb62e77add454764b15660f2ed6e36fd81c94ec"}
Jul 28 17:23:20.848693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.848672    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vpgqx" event={"ID":"50c96cb9-c926-46eb-8783-993ffbeb9e43","Type":"ContainerStarted","Data":"5fb4c77722091adfcc5daa00304b8436e40a5fbf836e9165e7939c5570759cc4"}
Jul 28 17:23:20.851938 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.851894    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gmzpv" podStartSLOduration=3.391852 podStartE2EDuration="21.851859342s" totalImagesPullingTime="18.460007342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:20.851501485 +0000 UTC m=+22.797240652" watchObservedRunningTime="2026-07-28 17:23:20.851859342 +0000 UTC m=+22.797598505"
Jul 28 17:23:20.852099 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.852069    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-165.ec2.internal" podStartSLOduration=20.852061149 podStartE2EDuration="20.852061149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:23:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:03.821198923 +0000 UTC m=+5.766938094" watchObservedRunningTime="2026-07-28 17:23:20.852061149 +0000 UTC m=+22.797800314"
Jul 28 17:23:20.884703 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.884646    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vpgqx" podStartSLOduration=7.101915484 podStartE2EDuration="21.88462995s" totalImagesPullingTime="14.782714466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:20.884260712 +0000 UTC m=+22.829999879" watchObservedRunningTime="2026-07-28 17:23:20.88462995 +0000 UTC m=+22.830369108"
Jul 28 17:23:20.885846 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.885796    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pq4hj" podStartSLOduration=3.46197238 podStartE2EDuration="21.885781662s" totalImagesPullingTime="18.423809282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:20.868989674 +0000 UTC m=+22.814728842" watchObservedRunningTime="2026-07-28 17:23:20.885781662 +0000 UTC m=+22.831520827"
Jul 28 17:23:20.933765 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.933702    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mt4rt" podStartSLOduration=7.149658263 podStartE2EDuration="21.933679719s" totalImagesPullingTime="14.784021456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:20.93313426 +0000 UTC m=+22.878873425" watchObservedRunningTime="2026-07-28 17:23:20.933679719 +0000 UTC m=+22.879418886"
Jul 28 17:23:20.952177 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:20.952127    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pt279" podStartSLOduration=3.405652632 podStartE2EDuration="21.952110688s" totalImagesPullingTime="18.546458056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:20.951944719 +0000 UTC m=+22.897683908" watchObservedRunningTime="2026-07-28 17:23:20.952110688 +0000 UTC m=+22.897849852"
Jul 28 17:23:21.584989 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:21.584960    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:21.585176 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:21.585105    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:21.698822 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:21.698784    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 17:23:21.852672 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:21.852591    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s" event={"ID":"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8","Type":"ContainerStarted","Data":"ca8f7324dfed67c69e2d4ea415c059108e9b6a7b0bc0d8d64db7bd808ca8cbb2"}
Jul 28 17:23:21.854190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:21.854150    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qldwg" event={"ID":"2d1a5dac-488b-405b-9045-04bd32406da9","Type":"ContainerStarted","Data":"56394d2d8c598f658045ee5e2066bf5785e5c738d796a89e6c83c9e0cd56023a"}
Jul 28 17:23:22.492724 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:22.492459    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:23:21.698808223Z","UUID":"db18dc28-5600-427e-acb6-50b06d0ce92e","Handler":null,"Name":"","Endpoint":""}
Jul 28 17:23:22.494351 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:22.494319    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 17:23:22.494486 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:22.494372    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 17:23:22.585304 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:22.585275    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:22.585304 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:22.585292    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:22.585545 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:22.585420    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:22.585603 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:22.585541    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:23.584624 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:23.584554    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:23.584982 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:23.584674    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:23.862064 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:23.861966    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s" event={"ID":"c4fcdfe7-9b79-4cca-b06c-36c56ba51af8","Type":"ContainerStarted","Data":"091520da8b858153bd003ac4caa7432f479ad3523affbffa6cf96f22262d9d13"}
Jul 28 17:23:23.865415 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:23.865386    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"84564054aeaaff6a03761bcc606bab90b0eb2dd3e0a0857e687d69b5e68accf0"}
Jul 28 17:23:23.881834 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:23.881791    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qldwg" podStartSLOduration=6.447046146 podStartE2EDuration="24.881775883s" totalImagesPullingTime="18.434729737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:21.873167306 +0000 UTC m=+23.818906472" watchObservedRunningTime="2026-07-28 17:23:23.881775883 +0000 UTC m=+25.827515086"
Jul 28 17:23:23.882303 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:23.882271    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bjv2s" podStartSLOduration=3.660061356 podStartE2EDuration="24.882259629s" totalImagesPullingTime="21.222198273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:23.881677076 +0000 UTC m=+25.827416243" watchObservedRunningTime="2026-07-28 17:23:23.882259629 +0000 UTC m=+25.827998795"
Jul 28 17:23:24.584438 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:24.584243    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:24.584621 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:24.584251    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:24.584621 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:24.584552    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:24.584621 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:24.584612    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:25.584657 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.584408    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:25.584787 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:25.584671    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:25.767955 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.767917    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:25.769851 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.768981    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:25.873561 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.873457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" event={"ID":"7f20edee-328b-4193-87de-90ad2efc3bb7","Type":"ContainerStarted","Data":"0e8db4f5196b6ec1df5df93836e599f6bf72533ba70c5f7e032960326e7278e9"}
Jul 28 17:23:25.873741 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.873698    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:25.873806 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.873766    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:25.875258 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.875229    2556 generic.go:356] "Generic (PLEG): container finished" podID="5e969c14-a4ea-49f0-ae8b-db7353c035ce" containerID="ae66172cf0a5e1db62ed780180d2e463589177093555682f9bcec1ff714ba066" exitCode=0
Jul 28 17:23:25.875374 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.875308    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerDied","Data":"ae66172cf0a5e1db62ed780180d2e463589177093555682f9bcec1ff714ba066"}
Jul 28 17:23:25.875522 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.875496    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:25.876002 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.875931    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mt4rt"
Jul 28 17:23:25.890013 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.889988    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:25.899523 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:25.899476    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt" podStartSLOduration=8.416768174 podStartE2EDuration="26.899460167s" totalImagesPullingTime="18.482691993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:25.898445801 +0000 UTC m=+27.844184965" watchObservedRunningTime="2026-07-28 17:23:25.899460167 +0000 UTC m=+27.845199357"
Jul 28 17:23:26.585188 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:26.585155    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:26.585675 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:26.585155    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:26.585675 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:26.585305    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:26.585675 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:26.585324    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:26.878351 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:26.878314    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:26.896737 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:26.896707    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:27.432229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.432194    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6gtv8"]
Jul 28 17:23:27.432402 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.432373    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:27.432490 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:27.432470    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:27.435448 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.435423    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nlpg9"]
Jul 28 17:23:27.435577 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.435523    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:27.435616 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:27.435601    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:27.446099 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.446073    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t7pdd"]
Jul 28 17:23:27.446206 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.446189    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:27.446313 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:27.446293    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:27.881573 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.881535    2556 generic.go:356] "Generic (PLEG): container finished" podID="5e969c14-a4ea-49f0-ae8b-db7353c035ce" containerID="a61be464d0ba7bdf9cdd9dc47b8004567040314d426b0a1ab37b0eef075a1340" exitCode=0
Jul 28 17:23:27.881998 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:27.881597    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerDied","Data":"a61be464d0ba7bdf9cdd9dc47b8004567040314d426b0a1ab37b0eef075a1340"}
Jul 28 17:23:28.585206 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:28.585177    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:28.585417 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:28.585277    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:29.585360 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:29.585118    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:29.585772 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:29.585158    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:29.585772 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:29.585467    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:29.585772 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:29.585507    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:29.887771 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:29.887673    2556 generic.go:356] "Generic (PLEG): container finished" podID="5e969c14-a4ea-49f0-ae8b-db7353c035ce" containerID="8d53e58b2c4e04976731dfa16d2fc0c28c38f13b51cf35196eb9411439021edf" exitCode=0
Jul 28 17:23:29.887771 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:29.887735    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerDied","Data":"8d53e58b2c4e04976731dfa16d2fc0c28c38f13b51cf35196eb9411439021edf"}
Jul 28 17:23:30.585101 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:30.585070    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:30.585363 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:30.585174    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:31.585029 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:31.584995    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:31.585563 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:31.585008    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:31.585563 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:31.585114    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6gtv8" podUID="962ce334-76b4-49da-9402-ba293e765556"
Jul 28 17:23:31.585563 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:31.585204    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nlpg9" podUID="6ae9655b-4180-4a82-9fd7-9eff9b69d437"
Jul 28 17:23:32.183711 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.183674    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:32.183921 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.183801    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:32.183921 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.183860    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:04.18384249 +0000 UTC m=+66.129581632 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:23:32.284007 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.283971    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:32.284197 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.284036    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:32.284197 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.284148    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:32.284311 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.284228    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret podName:a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:04.284207151 +0000 UTC m=+66.229946313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret") pod "global-pull-secret-syncer-t7pdd" (UID: "a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:23:32.284311 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.284166    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:23:32.284311 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.284267    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:23:32.284311 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.284280    2556 projected.go:196] Error preparing data for projected volume kube-api-access-nww5l for pod openshift-network-diagnostics/network-check-target-6gtv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:32.284494 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.284334    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l podName:962ce334-76b4-49da-9402-ba293e765556 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:04.284319287 +0000 UTC m=+66.230058435 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nww5l" (UniqueName: "kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l") pod "network-check-target-6gtv8" (UID: "962ce334-76b4-49da-9402-ba293e765556") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:23:32.584791 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.584761    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:32.584937 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:32.584911    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7pdd" podUID="a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7"
Jul 28 17:23:32.815286 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.815258    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-165.ec2.internal" event="NodeReady"
Jul 28 17:23:32.815735 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.815414    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 17:23:32.867528 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.867446    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b8tjb"]
Jul 28 17:23:32.870449 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.870416    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xgv75"]
Jul 28 17:23:32.870618 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.870563    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:32.872324 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.872303    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:32.873084 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.873055    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 17:23:32.873193 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.873125    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 17:23:32.873548 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.873526    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fqthr\""
Jul 28 17:23:32.874285 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.874254    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 17:23:32.874524 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.874506    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 17:23:32.874612 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.874527    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 17:23:32.874745 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.874721    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h4rhp\""
Jul 28 17:23:32.883216 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.883156    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b8tjb"]
Jul 28 17:23:32.883785 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.883763    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xgv75"]
Jul 28 17:23:32.988890 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.988844    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:32.989053 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.988915    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b63c8dfa-9627-4721-997b-ae04c040ff9a-tmp-dir\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:32.989053 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.988996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2qjj\" (UniqueName: \"kubernetes.io/projected/017a2f4a-4f71-435a-9d86-3e2e168787fd-kube-api-access-w2qjj\") pod \"ingress-canary-xgv75\" (UID: \"017a2f4a-4f71-435a-9d86-3e2e168787fd\") " pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:32.989053 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.989031    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b63c8dfa-9627-4721-997b-ae04c040ff9a-config-volume\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:32.989053 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.989053    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5v7j\" (UniqueName: \"kubernetes.io/projected/b63c8dfa-9627-4721-997b-ae04c040ff9a-kube-api-access-t5v7j\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:32.989212 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:32.989070    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/017a2f4a-4f71-435a-9d86-3e2e168787fd-cert\") pod \"ingress-canary-xgv75\" (UID: \"017a2f4a-4f71-435a-9d86-3e2e168787fd\") " pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:33.089802 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.089767    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2qjj\" (UniqueName: \"kubernetes.io/projected/017a2f4a-4f71-435a-9d86-3e2e168787fd-kube-api-access-w2qjj\") pod \"ingress-canary-xgv75\" (UID: \"017a2f4a-4f71-435a-9d86-3e2e168787fd\") " pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:33.089802 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.089809    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b63c8dfa-9627-4721-997b-ae04c040ff9a-config-volume\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.090045 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.089832    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5v7j\" (UniqueName: \"kubernetes.io/projected/b63c8dfa-9627-4721-997b-ae04c040ff9a-kube-api-access-t5v7j\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.090045 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.089854    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/017a2f4a-4f71-435a-9d86-3e2e168787fd-cert\") pod \"ingress-canary-xgv75\" (UID: \"017a2f4a-4f71-435a-9d86-3e2e168787fd\") " pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:33.090045 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.089941    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.090045 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.089983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b63c8dfa-9627-4721-997b-ae04c040ff9a-tmp-dir\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.090236 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:33.090145    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:23:33.090236 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:33.090217    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls podName:b63c8dfa-9627-4721-997b-ae04c040ff9a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:33.590194291 +0000 UTC m=+35.535933433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls") pod "dns-default-b8tjb" (UID: "b63c8dfa-9627-4721-997b-ae04c040ff9a") : secret "dns-default-metrics-tls" not found
Jul 28 17:23:33.090430 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.090399    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b63c8dfa-9627-4721-997b-ae04c040ff9a-tmp-dir\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.090578 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.090558    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b63c8dfa-9627-4721-997b-ae04c040ff9a-config-volume\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.094689 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.094667    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/017a2f4a-4f71-435a-9d86-3e2e168787fd-cert\") pod \"ingress-canary-xgv75\" (UID: \"017a2f4a-4f71-435a-9d86-3e2e168787fd\") " pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:33.099513 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.099471    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5v7j\" (UniqueName: \"kubernetes.io/projected/b63c8dfa-9627-4721-997b-ae04c040ff9a-kube-api-access-t5v7j\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.099706 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.099683    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2qjj\" (UniqueName: \"kubernetes.io/projected/017a2f4a-4f71-435a-9d86-3e2e168787fd-kube-api-access-w2qjj\") pod \"ingress-canary-xgv75\" (UID: \"017a2f4a-4f71-435a-9d86-3e2e168787fd\") " pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:33.190740 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.190704    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xgv75"
Jul 28 17:23:33.584339 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.584259    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:23:33.584504 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.584259    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:23:33.587794 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.587772    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ztptx\""
Jul 28 17:23:33.587794 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.587787    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:23:33.588064 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.587787    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:23:33.588064 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.587773    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:23:33.588064 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.587805    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-27qqv\""
Jul 28 17:23:33.593962 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:33.593940    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:33.594138 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:33.594116    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:23:33.594249 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:33.594181    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls podName:b63c8dfa-9627-4721-997b-ae04c040ff9a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:34.594163118 +0000 UTC m=+36.539902263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls") pod "dns-default-b8tjb" (UID: "b63c8dfa-9627-4721-997b-ae04c040ff9a") : secret "dns-default-metrics-tls" not found
Jul 28 17:23:34.168013 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:34.167769    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xgv75"]
Jul 28 17:23:34.171680 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:34.171649    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017a2f4a_4f71_435a_9d86_3e2e168787fd.slice/crio-7c25e529c6927e81ca7710e5c8a4ada96ef2dcc9443815df3310f055afcd543b WatchSource:0}: Error finding container 7c25e529c6927e81ca7710e5c8a4ada96ef2dcc9443815df3310f055afcd543b: Status 404 returned error can't find the container with id 7c25e529c6927e81ca7710e5c8a4ada96ef2dcc9443815df3310f055afcd543b
Jul 28 17:23:34.585150 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:34.585112    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:23:34.587645 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:34.587622    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:23:34.603339 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:34.603310    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:34.603475 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:34.603457    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:23:34.603526 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:34.603517    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls podName:b63c8dfa-9627-4721-997b-ae04c040ff9a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:36.603502143 +0000 UTC m=+38.549241286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls") pod "dns-default-b8tjb" (UID: "b63c8dfa-9627-4721-997b-ae04c040ff9a") : secret "dns-default-metrics-tls" not found
Jul 28 17:23:34.900851 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:34.900774    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xgv75" event={"ID":"017a2f4a-4f71-435a-9d86-3e2e168787fd","Type":"ContainerStarted","Data":"7c25e529c6927e81ca7710e5c8a4ada96ef2dcc9443815df3310f055afcd543b"}
Jul 28 17:23:34.903844 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:34.903811    2556 generic.go:356] "Generic (PLEG): container finished" podID="5e969c14-a4ea-49f0-ae8b-db7353c035ce" containerID="eb36d4d87c4df6c09dbcba37ea00969ed8803695dea84c2323de5c8a1d95795c" exitCode=0
Jul 28 17:23:34.903988 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:34.903863    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerDied","Data":"eb36d4d87c4df6c09dbcba37ea00969ed8803695dea84c2323de5c8a1d95795c"}
Jul 28 17:23:35.910338 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:35.910308    2556 generic.go:356] "Generic (PLEG): container finished" podID="5e969c14-a4ea-49f0-ae8b-db7353c035ce" containerID="e2a24b37a2a6788b8c252990099abdd6eab086b448a1f638a0d4ebf805f0999d" exitCode=0
Jul 28 17:23:35.910857 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:35.910361    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerDied","Data":"e2a24b37a2a6788b8c252990099abdd6eab086b448a1f638a0d4ebf805f0999d"}
Jul 28 17:23:36.618854 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:36.618823    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:36.619011 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:36.618993    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:23:36.619071 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:36.619061    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls podName:b63c8dfa-9627-4721-997b-ae04c040ff9a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:40.61904493 +0000 UTC m=+42.564784076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls") pod "dns-default-b8tjb" (UID: "b63c8dfa-9627-4721-997b-ae04c040ff9a") : secret "dns-default-metrics-tls" not found
Jul 28 17:23:36.915705 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:36.915536    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-th8j5" event={"ID":"5e969c14-a4ea-49f0-ae8b-db7353c035ce","Type":"ContainerStarted","Data":"d702c16cda1598b44d84cb916285c8b428d30ca142129ef3cf761f5a666ae816"}
Jul 28 17:23:36.939607 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:36.939550    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-th8j5" podStartSLOduration=8.409965446 podStartE2EDuration="37.939529177s" totalImagesPullingTime="29.529563731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:36.93791573 +0000 UTC m=+38.883654919" watchObservedRunningTime="2026-07-28 17:23:36.939529177 +0000 UTC m=+38.885268342"
Jul 28 17:23:37.919568 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:37.919533    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xgv75" event={"ID":"017a2f4a-4f71-435a-9d86-3e2e168787fd","Type":"ContainerStarted","Data":"a5002b1f2498cab64f282d0551a246b9abefb2e17e21348ced6cb870451490b3"}
Jul 28 17:23:37.935600 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:37.935554    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xgv75" podStartSLOduration=3.245282411 podStartE2EDuration="5.935538995s" totalImagesPullingTime="2.690256584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:37.934912439 +0000 UTC m=+39.880651603" watchObservedRunningTime="2026-07-28 17:23:37.935538995 +0000 UTC m=+39.881278156"
Jul 28 17:23:40.646037 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:40.645988    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:40.646386 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:40.646135    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:23:40.646386 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:40.646198    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls podName:b63c8dfa-9627-4721-997b-ae04c040ff9a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:48.646183852 +0000 UTC m=+50.591922994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls") pod "dns-default-b8tjb" (UID: "b63c8dfa-9627-4721-997b-ae04c040ff9a") : secret "dns-default-metrics-tls" not found
Jul 28 17:23:48.703380 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:48.703341    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:23:48.703899 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:48.703498    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:23:48.703899 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:48.703577    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls podName:b63c8dfa-9627-4721-997b-ae04c040ff9a nodeName:}" failed. No retries permitted until 2026-07-28 17:24:04.703560386 +0000 UTC m=+66.649299528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls") pod "dns-default-b8tjb" (UID: "b63c8dfa-9627-4721-997b-ae04c040ff9a") : secret "dns-default-metrics-tls" not found
Jul 28 17:23:54.180210 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.180175    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"]
Jul 28 17:23:54.182515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.182499    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"
Jul 28 17:23:54.184630 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.184609    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:23:54.184740 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.184687    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 17:23:54.185325 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.185309    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ctxwx\""
Jul 28 17:23:54.205824 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.205794    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"]
Jul 28 17:23:54.240583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.240550    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6nt4\" (UniqueName: \"kubernetes.io/projected/eb9fa04d-1679-4d4e-bef8-77cdb95e07f8-kube-api-access-z6nt4\") pod \"volume-data-source-validator-549c6d54cf-crtrt\" (UID: \"eb9fa04d-1679-4d4e-bef8-77cdb95e07f8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"
Jul 28 17:23:54.258594 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.258558    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bgkjm"]
Jul 28 17:23:54.260339 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.260316    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.262526 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.262504    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"]
Jul 28 17:23:54.264260 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.264241    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.270453 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.270437    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 17:23:54.270555 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.270511    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nvbms\""
Jul 28 17:23:54.270765 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.270749    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 17:23:54.270858 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.270845    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 17:23:54.271156 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.271143    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sxcrx\""
Jul 28 17:23:54.271278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.271265    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 17:23:54.272210 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.272193    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 17:23:54.272290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.272214    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 17:23:54.272765 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.272752    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 17:23:54.281111 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.281094    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 17:23:54.293372 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.293348    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bgkjm"]
Jul 28 17:23:54.294061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.294040    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"]
Jul 28 17:23:54.301522 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.301501    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 17:23:54.333484 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.333456    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"]
Jul 28 17:23:54.335178 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.335162    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.338095 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.338071    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"]
Jul 28 17:23:54.339004 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.338979    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 17:23:54.339169 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.339144    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zzvkr\""
Jul 28 17:23:54.339501 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.339485    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 17:23:54.339771 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.339750    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 17:23:54.339883 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.339789    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:23:54.341384 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341366    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.341479 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341392    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19a545db-ab73-4366-b684-bb8c31012857-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.341479 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341414    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6nt4\" (UniqueName: \"kubernetes.io/projected/eb9fa04d-1679-4d4e-bef8-77cdb95e07f8-kube-api-access-z6nt4\") pod \"volume-data-source-validator-549c6d54cf-crtrt\" (UID: \"eb9fa04d-1679-4d4e-bef8-77cdb95e07f8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"
Jul 28 17:23:54.341479 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341441    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a3217ae-e76f-4064-bff2-e52f9c649912-service-ca-bundle\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.341635 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341571    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a3217ae-e76f-4064-bff2-e52f9c649912-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.341635 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341604    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tp6d\" (UniqueName: \"kubernetes.io/projected/8a3217ae-e76f-4064-bff2-e52f9c649912-kube-api-access-4tp6d\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.341635 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341609    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:54.341776 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341651    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xczn4\" (UniqueName: \"kubernetes.io/projected/19a545db-ab73-4366-b684-bb8c31012857-kube-api-access-xczn4\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.341776 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341677    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8a3217ae-e76f-4064-bff2-e52f9c649912-snapshots\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.341776 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341711    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwwx\" (UniqueName: \"kubernetes.io/projected/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-kube-api-access-xwwwx\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.341776 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341736    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.341776 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341763    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a3217ae-e76f-4064-bff2-e52f9c649912-tmp\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.341993 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341798    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.341993 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.341822    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a3217ae-e76f-4064-bff2-e52f9c649912-serving-cert\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.343846 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.343826    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 17:23:54.343986 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.343940    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 17:23:54.343986 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.343833    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5vtd5\""
Jul 28 17:23:54.344214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.344199    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:23:54.352617 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.352596    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"]
Jul 28 17:23:54.375892 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.370200    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"]
Jul 28 17:23:54.379539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.379506    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6nt4\" (UniqueName: \"kubernetes.io/projected/eb9fa04d-1679-4d4e-bef8-77cdb95e07f8-kube-api-access-z6nt4\") pod \"volume-data-source-validator-549c6d54cf-crtrt\" (UID: \"eb9fa04d-1679-4d4e-bef8-77cdb95e07f8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"
Jul 28 17:23:54.431640 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.431540    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-46tzq"]
Jul 28 17:23:54.433579 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.433562    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5996b98fbf-2jsq8"]
Jul 28 17:23:54.433694 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.433676    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.435421 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.435404    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.438854 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.438835    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 17:23:54.439060 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.439044    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 17:23:54.439151 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.439095    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:23:54.439353 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.439337    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rvwrb\""
Jul 28 17:23:54.439437 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.439349    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 17:23:54.439749 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.439730    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7d6vn\""
Jul 28 17:23:54.439883 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.439855    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 17:23:54.440116 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.440100    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 17:23:54.440313 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.440296    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 17:23:54.442278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a3217ae-e76f-4064-bff2-e52f9c649912-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.442362 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tp6d\" (UniqueName: \"kubernetes.io/projected/8a3217ae-e76f-4064-bff2-e52f9c649912-kube-api-access-4tp6d\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.442362 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442327    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:54.442362 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442356    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6rqb\" (UniqueName: \"kubernetes.io/projected/fafd8869-525a-4770-9f01-c5d8bf5ea11a-kube-api-access-b6rqb\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:54.442510 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xczn4\" (UniqueName: \"kubernetes.io/projected/19a545db-ab73-4366-b684-bb8c31012857-kube-api-access-xczn4\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.442510 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442446    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8a3217ae-e76f-4064-bff2-e52f9c649912-snapshots\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.442609 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442478    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.442609 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442568    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwwx\" (UniqueName: \"kubernetes.io/projected/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-kube-api-access-xwwwx\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.442715 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442622    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a3217ae-e76f-4064-bff2-e52f9c649912-tmp\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.442715 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442656    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-trusted-ca\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.442805 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442724    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.442805 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442752    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a3217ae-e76f-4064-bff2-e52f9c649912-serving-cert\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.442805 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.442973 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442897    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d7e74f7-415b-4799-9ef3-3019f56ca233-ca-trust-extracted\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.442973 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.442917    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:54.442973 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442963    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9260f1e-1598-4199-8917-009e5d328ae6-serving-cert\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.443095 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.442984    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a3217ae-e76f-4064-bff2-e52f9c649912-tmp\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.443095 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.442996    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls podName:19a545db-ab73-4366-b684-bb8c31012857 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:54.942978629 +0000 UTC m=+56.888717782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fv7pr" (UID: "19a545db-ab73-4366-b684-bb8c31012857") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:54.443095 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443016    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d9260f1e-1598-4199-8917-009e5d328ae6-config\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.443252 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443189    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a3217ae-e76f-4064-bff2-e52f9c649912-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.443431 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443406    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9260f1e-1598-4199-8917-009e5d328ae6-trusted-ca\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.443527 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443446    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj798\" (UniqueName: \"kubernetes.io/projected/d9260f1e-1598-4199-8917-009e5d328ae6-kube-api-access-nj798\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.443527 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443443    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8a3217ae-e76f-4064-bff2-e52f9c649912-snapshots\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.443527 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443486    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a3217ae-e76f-4064-bff2-e52f9c649912-service-ca-bundle\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.443677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443533    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-installation-pull-secrets\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.443677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443579    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-image-registry-private-configuration\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.443677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443599    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.443677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443608    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.443862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-bound-sa-token\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.443862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443785    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-certificates\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.443862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443813    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxrvj\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-kube-api-access-fxrvj\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.443862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.443842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19a545db-ab73-4366-b684-bb8c31012857-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.444101 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.444081    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a3217ae-e76f-4064-bff2-e52f9c649912-service-ca-bundle\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.445479 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.445453    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19a545db-ab73-4366-b684-bb8c31012857-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.446083 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.446060    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a3217ae-e76f-4064-bff2-e52f9c649912-serving-cert\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.446440 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.446419    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.447927 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.447854    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 17:23:54.448276 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.448251    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-46tzq"]
Jul 28 17:23:54.448413 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.448303    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 17:23:54.453150 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.453132    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5996b98fbf-2jsq8"]
Jul 28 17:23:54.459927 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.459865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwwx\" (UniqueName: \"kubernetes.io/projected/42149cf6-a57a-4cbb-ae05-81d2bd5ac36e-kube-api-access-xwwwx\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4\" (UID: \"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.460020 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.460000    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xczn4\" (UniqueName: \"kubernetes.io/projected/19a545db-ab73-4366-b684-bb8c31012857-kube-api-access-xczn4\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.460926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.460910    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tp6d\" (UniqueName: \"kubernetes.io/projected/8a3217ae-e76f-4064-bff2-e52f9c649912-kube-api-access-4tp6d\") pod \"insights-operator-5885fdb759-bgkjm\" (UID: \"8a3217ae-e76f-4064-bff2-e52f9c649912\") " pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.491759 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.491726    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"
Jul 28 17:23:54.545125 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545089    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-image-registry-private-configuration\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545281 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545139    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545281 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-bound-sa-token\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545281 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545233    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-certificates\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545275    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxrvj\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-kube-api-access-fxrvj\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545351    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:54.545446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545378    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6rqb\" (UniqueName: \"kubernetes.io/projected/fafd8869-525a-4770-9f01-c5d8bf5ea11a-kube-api-access-b6rqb\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:54.545653 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.545581    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:23:54.545653 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.545593    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:23:54.545653 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.545600    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5996b98fbf-2jsq8: secret "image-registry-tls" not found
Jul 28 17:23:54.545653 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-trusted-ca\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.545674    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls podName:fafd8869-525a-4770-9f01-c5d8bf5ea11a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:55.045639125 +0000 UTC m=+56.991378269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls") pod "cluster-samples-operator-6748489656-cprmc" (UID: "fafd8869-525a-4770-9f01-c5d8bf5ea11a") : secret "samples-operator-tls" not found
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.545701    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls podName:8d7e74f7-415b-4799-9ef3-3019f56ca233 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:55.045688613 +0000 UTC m=+56.991427763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls") pod "image-registry-5996b98fbf-2jsq8" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233") : secret "image-registry-tls" not found
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545731    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d7e74f7-415b-4799-9ef3-3019f56ca233-ca-trust-extracted\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9260f1e-1598-4199-8917-009e5d328ae6-serving-cert\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545806    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d9260f1e-1598-4199-8917-009e5d328ae6-config\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545831    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9260f1e-1598-4199-8917-009e5d328ae6-trusted-ca\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-certificates\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.545889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545861    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj798\" (UniqueName: \"kubernetes.io/projected/d9260f1e-1598-4199-8917-009e5d328ae6-kube-api-access-nj798\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.546200 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.545956    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-installation-pull-secrets\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.547385 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.547070    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d9260f1e-1598-4199-8917-009e5d328ae6-config\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.547385 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.547195    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9260f1e-1598-4199-8917-009e5d328ae6-trusted-ca\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.547385 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.547308    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-trusted-ca\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.547385 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.547368    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d7e74f7-415b-4799-9ef3-3019f56ca233-ca-trust-extracted\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.549947 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.548480    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-image-registry-private-configuration\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.554609 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.554538    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9260f1e-1598-4199-8917-009e5d328ae6-serving-cert\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.555273 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.554912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-installation-pull-secrets\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.558109 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.558084    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-bound-sa-token\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.558754 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.558707    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6rqb\" (UniqueName: \"kubernetes.io/projected/fafd8869-525a-4770-9f01-c5d8bf5ea11a-kube-api-access-b6rqb\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:54.559349 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.559298    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxrvj\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-kube-api-access-fxrvj\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:54.559440 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.559416    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj798\" (UniqueName: \"kubernetes.io/projected/d9260f1e-1598-4199-8917-009e5d328ae6-kube-api-access-nj798\") pod \"console-operator-584dcb6b54-46tzq\" (UID: \"d9260f1e-1598-4199-8917-009e5d328ae6\") " pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.570195 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.570175    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bgkjm"
Jul 28 17:23:54.618678 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.618632    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt"]
Jul 28 17:23:54.622760 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:54.622736    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9fa04d_1679_4d4e_bef8_77cdb95e07f8.slice/crio-71f9a59da58b50c78670dc64e6e81dfda36d11a986896dfb0724c9b67c93e519 WatchSource:0}: Error finding container 71f9a59da58b50c78670dc64e6e81dfda36d11a986896dfb0724c9b67c93e519: Status 404 returned error can't find the container with id 71f9a59da58b50c78670dc64e6e81dfda36d11a986896dfb0724c9b67c93e519
Jul 28 17:23:54.648344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.648318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"
Jul 28 17:23:54.701290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.701091    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bgkjm"]
Jul 28 17:23:54.705331 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:54.705302    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a3217ae_e76f_4064_bff2_e52f9c649912.slice/crio-cb0aa98a5853023ab6a79ecc23b56250cf6af0cb467e20de2d26806316502dfc WatchSource:0}: Error finding container cb0aa98a5853023ab6a79ecc23b56250cf6af0cb467e20de2d26806316502dfc: Status 404 returned error can't find the container with id cb0aa98a5853023ab6a79ecc23b56250cf6af0cb467e20de2d26806316502dfc
Jul 28 17:23:54.745255 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.745225    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:23:54.774006 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.773968    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4"]
Jul 28 17:23:54.777281 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:54.777249    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42149cf6_a57a_4cbb_ae05_81d2bd5ac36e.slice/crio-991520f299ad6ffbd0598aca3e29700ca7184279803a325407118f7011c1afcd WatchSource:0}: Error finding container 991520f299ad6ffbd0598aca3e29700ca7184279803a325407118f7011c1afcd: Status 404 returned error can't find the container with id 991520f299ad6ffbd0598aca3e29700ca7184279803a325407118f7011c1afcd
Jul 28 17:23:54.872055 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.872026    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-46tzq"]
Jul 28 17:23:54.876276 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:23:54.876246    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9260f1e_1598_4199_8917_009e5d328ae6.slice/crio-f122b178eec2214256a1b6e549ef65bd2a37f2b5af73e9b1e756bc3119fce8c0 WatchSource:0}: Error finding container f122b178eec2214256a1b6e549ef65bd2a37f2b5af73e9b1e756bc3119fce8c0: Status 404 returned error can't find the container with id f122b178eec2214256a1b6e549ef65bd2a37f2b5af73e9b1e756bc3119fce8c0
Jul 28 17:23:54.950996 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.950903    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:54.951143 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.951062    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:54.951143 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:54.951135    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls podName:19a545db-ab73-4366-b684-bb8c31012857 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:55.951113767 +0000 UTC m=+57.896852909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fv7pr" (UID: "19a545db-ab73-4366-b684-bb8c31012857") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:54.959618 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.959590    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bgkjm" event={"ID":"8a3217ae-e76f-4064-bff2-e52f9c649912","Type":"ContainerStarted","Data":"cb0aa98a5853023ab6a79ecc23b56250cf6af0cb467e20de2d26806316502dfc"}
Jul 28 17:23:54.960547 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.960526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" event={"ID":"d9260f1e-1598-4199-8917-009e5d328ae6","Type":"ContainerStarted","Data":"f122b178eec2214256a1b6e549ef65bd2a37f2b5af73e9b1e756bc3119fce8c0"}
Jul 28 17:23:54.961399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.961383    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4" event={"ID":"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e","Type":"ContainerStarted","Data":"991520f299ad6ffbd0598aca3e29700ca7184279803a325407118f7011c1afcd"}
Jul 28 17:23:54.962373 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:54.962357    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt" event={"ID":"eb9fa04d-1679-4d4e-bef8-77cdb95e07f8","Type":"ContainerStarted","Data":"71f9a59da58b50c78670dc64e6e81dfda36d11a986896dfb0724c9b67c93e519"}
Jul 28 17:23:55.051821 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:55.051790    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:55.051981 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:55.051839    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:55.051981 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:55.051960    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:23:55.052060 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:55.052022    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls podName:fafd8869-525a-4770-9f01-c5d8bf5ea11a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:56.052004386 +0000 UTC m=+57.997743528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls") pod "cluster-samples-operator-6748489656-cprmc" (UID: "fafd8869-525a-4770-9f01-c5d8bf5ea11a") : secret "samples-operator-tls" not found
Jul 28 17:23:55.052060 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:55.051964    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:23:55.052060 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:55.052056    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5996b98fbf-2jsq8: secret "image-registry-tls" not found
Jul 28 17:23:55.052165 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:55.052095    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls podName:8d7e74f7-415b-4799-9ef3-3019f56ca233 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:56.052081171 +0000 UTC m=+57.997820326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls") pod "image-registry-5996b98fbf-2jsq8" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233") : secret "image-registry-tls" not found
Jul 28 17:23:55.962487 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:55.962455    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:55.962901 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:55.962655    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:55.962901 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:55.962718    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls podName:19a545db-ab73-4366-b684-bb8c31012857 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:57.962702779 +0000 UTC m=+59.908441920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fv7pr" (UID: "19a545db-ab73-4366-b684-bb8c31012857") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:55.967596 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:55.967421    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt" event={"ID":"eb9fa04d-1679-4d4e-bef8-77cdb95e07f8","Type":"ContainerStarted","Data":"09718c7b8098fdf3da89edd8d79de262937eb8098792596089aeb68565248e8b"}
Jul 28 17:23:55.985709 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:55.985662    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-crtrt" podStartSLOduration=0.729007824 podStartE2EDuration="1.985645535s" totalImagesPullingTime="1.256637711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:55.984581491 +0000 UTC m=+57.930320656" watchObservedRunningTime="2026-07-28 17:23:55.985645535 +0000 UTC m=+57.931384699"
Jul 28 17:23:56.063202 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:56.063167    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:56.063381 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:56.063307    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:56.063549 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:56.063525    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:23:56.063660 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:56.063590    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls podName:fafd8869-525a-4770-9f01-c5d8bf5ea11a nodeName:}" failed. No retries permitted until 2026-07-28 17:23:58.063574254 +0000 UTC m=+60.009313400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls") pod "cluster-samples-operator-6748489656-cprmc" (UID: "fafd8869-525a-4770-9f01-c5d8bf5ea11a") : secret "samples-operator-tls" not found
Jul 28 17:23:56.063752 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:56.063692    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:23:56.063752 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:56.063712    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5996b98fbf-2jsq8: secret "image-registry-tls" not found
Jul 28 17:23:56.063854 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:56.063765    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls podName:8d7e74f7-415b-4799-9ef3-3019f56ca233 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:58.063748569 +0000 UTC m=+60.009487716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls") pod "image-registry-5996b98fbf-2jsq8" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233") : secret "image-registry-tls" not found
Jul 28 17:23:57.974438 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:57.974411    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/0.log"
Jul 28 17:23:57.974928 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:57.974452    2556 generic.go:356] "Generic (PLEG): container finished" podID="d9260f1e-1598-4199-8917-009e5d328ae6" containerID="7f90e954a012933b3eebadd505bd01812d047fd8c06a570c68bec9ce201a082c" exitCode=255
Jul 28 17:23:57.974928 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:57.974530    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" event={"ID":"d9260f1e-1598-4199-8917-009e5d328ae6","Type":"ContainerDied","Data":"7f90e954a012933b3eebadd505bd01812d047fd8c06a570c68bec9ce201a082c"}
Jul 28 17:23:57.974928 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:57.974815    2556 scope.go:117] "RemoveContainer" containerID="7f90e954a012933b3eebadd505bd01812d047fd8c06a570c68bec9ce201a082c"
Jul 28 17:23:57.976154 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:57.976125    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4" event={"ID":"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e","Type":"ContainerStarted","Data":"36d49e0995e2138cd9f73ea035c612f884aad92e4d84a0a37c100c1631507fa3"}
Jul 28 17:23:57.977672 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:57.977648    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bgkjm" event={"ID":"8a3217ae-e76f-4064-bff2-e52f9c649912","Type":"ContainerStarted","Data":"18601024b27a7effe5568d1cfa7b8ae21f9631815a57bf696c6e02902ece95f8"}
Jul 28 17:23:57.979183 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:57.979159    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:23:57.979354 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:57.979337    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:57.979426 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:57.979415    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls podName:19a545db-ab73-4366-b684-bb8c31012857 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:01.979391676 +0000 UTC m=+63.925130825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fv7pr" (UID: "19a545db-ab73-4366-b684-bb8c31012857") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:23:58.014204 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.014147    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-bgkjm" podStartSLOduration=1.092953852 podStartE2EDuration="4.01412558s" totalImagesPullingTime="2.921171728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:58.012899734 +0000 UTC m=+59.958638897" watchObservedRunningTime="2026-07-28 17:23:58.01412558 +0000 UTC m=+59.959864745"
Jul 28 17:23:58.029603 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.029550    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4" podStartSLOduration=1.183319775 podStartE2EDuration="4.029497965s" totalImagesPullingTime="2.84617819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:58.029190333 +0000 UTC m=+59.974929498" watchObservedRunningTime="2026-07-28 17:23:58.029497965 +0000 UTC m=+59.975237131"
Jul 28 17:23:58.080508 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.080469    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:23:58.080938 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.080647    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:23:58.080938 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.080670    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5996b98fbf-2jsq8: secret "image-registry-tls" not found
Jul 28 17:23:58.080938 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.080735    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls podName:8d7e74f7-415b-4799-9ef3-3019f56ca233 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:02.080714093 +0000 UTC m=+64.026453264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls") pod "image-registry-5996b98fbf-2jsq8" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233") : secret "image-registry-tls" not found
Jul 28 17:23:58.081161 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.080986    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:23:58.081222 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.081161    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:23:58.081222 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.081214    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls podName:fafd8869-525a-4770-9f01-c5d8bf5ea11a nodeName:}" failed. No retries permitted until 2026-07-28 17:24:02.081197845 +0000 UTC m=+64.026937006 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls") pod "cluster-samples-operator-6748489656-cprmc" (UID: "fafd8869-525a-4770-9f01-c5d8bf5ea11a") : secret "samples-operator-tls" not found
Jul 28 17:23:58.758259 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.758224    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k7br8"]
Jul 28 17:23:58.787243 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.787207    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k7br8"]
Jul 28 17:23:58.787382 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.787346    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.789465 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.789436    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 17:23:58.789614 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.789548    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hw8gz\""
Jul 28 17:23:58.789614 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.789555    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 17:23:58.887485 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.887457    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.887719 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.887501    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c364111e-5375-462e-93bd-cbd691f9b2ef-crio-socket\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.887719 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.887563    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c364111e-5375-462e-93bd-cbd691f9b2ef-data-volume\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.887719 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.887709    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsvk\" (UniqueName: \"kubernetes.io/projected/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-api-access-zlsvk\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.887866 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.887741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.900542 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.900480    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrdnt"
Jul 28 17:23:58.983270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.983246    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/1.log"
Jul 28 17:23:58.984196 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.984176    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/0.log"
Jul 28 17:23:58.984301 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.984216    2556 generic.go:356] "Generic (PLEG): container finished" podID="d9260f1e-1598-4199-8917-009e5d328ae6" containerID="10049bd1ba9bdd052cf6f3e3be0ce91db7757cded6f30676390145a690b20d51" exitCode=255
Jul 28 17:23:58.984343 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.984298    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" event={"ID":"d9260f1e-1598-4199-8917-009e5d328ae6","Type":"ContainerDied","Data":"10049bd1ba9bdd052cf6f3e3be0ce91db7757cded6f30676390145a690b20d51"}
Jul 28 17:23:58.984380 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.984354    2556 scope.go:117] "RemoveContainer" containerID="7f90e954a012933b3eebadd505bd01812d047fd8c06a570c68bec9ce201a082c"
Jul 28 17:23:58.984533 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.984515    2556 scope.go:117] "RemoveContainer" containerID="10049bd1ba9bdd052cf6f3e3be0ce91db7757cded6f30676390145a690b20d51"
Jul 28 17:23:58.984735 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.984712    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-46tzq_openshift-console-operator(d9260f1e-1598-4199-8917-009e5d328ae6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" podUID="d9260f1e-1598-4199-8917-009e5d328ae6"
Jul 28 17:23:58.988492 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.988470    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.988605 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.988506    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c364111e-5375-462e-93bd-cbd691f9b2ef-crio-socket\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.988605 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.988541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c364111e-5375-462e-93bd-cbd691f9b2ef-data-volume\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.988605 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.988586    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:23:58.988759 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.988605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c364111e-5375-462e-93bd-cbd691f9b2ef-crio-socket\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.988759 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.988662    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:23:59.488642292 +0000 UTC m=+61.434381450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : configmap "kube-rbac-proxy" not found
Jul 28 17:23:58.988759 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.988721    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsvk\" (UniqueName: \"kubernetes.io/projected/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-api-access-zlsvk\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.989190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.989160    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.989311 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.989296    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:23:58.989376 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:58.989366    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:23:59.489349469 +0000 UTC m=+61.435088631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:23:58.989456 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.989433    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c364111e-5375-462e-93bd-cbd691f9b2ef-data-volume\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:58.997074 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:58.997055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsvk\" (UniqueName: \"kubernetes.io/projected/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-api-access-zlsvk\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:59.493801 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:59.493764    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:59.494006 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:59.493825    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:23:59.494006 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:59.493922    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:23:59.494006 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:59.493927    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:23:59.494006 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:59.493998    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:00.493968568 +0000 UTC m=+62.439707710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : configmap "kube-rbac-proxy" not found
Jul 28 17:23:59.494006 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:59.494012    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:00.494006203 +0000 UTC m=+62.439745344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:23:59.987706 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:59.987678    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/1.log"
Jul 28 17:23:59.988184 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:23:59.988027    2556 scope.go:117] "RemoveContainer" containerID="10049bd1ba9bdd052cf6f3e3be0ce91db7757cded6f30676390145a690b20d51"
Jul 28 17:23:59.988184 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:23:59.988162    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-46tzq_openshift-console-operator(d9260f1e-1598-4199-8917-009e5d328ae6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" podUID="d9260f1e-1598-4199-8917-009e5d328ae6"
Jul 28 17:24:00.334677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.334604    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rwfpq"]
Jul 28 17:24:00.337436 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.337420    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.340287 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.340262    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 17:24:00.340287 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.340281    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 17:24:00.340466 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.340264    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-978ld\""
Jul 28 17:24:00.340466 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.340341    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 17:24:00.340466 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.340262    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 17:24:00.350536 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.350516    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rwfpq"]
Jul 28 17:24:00.399423 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.399387    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dqkk\" (UniqueName: \"kubernetes.io/projected/04c32f56-70a6-4641-ba00-6c87ca060417-kube-api-access-6dqkk\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.399595 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.399477    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/04c32f56-70a6-4641-ba00-6c87ca060417-signing-cabundle\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.399595 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.399507    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/04c32f56-70a6-4641-ba00-6c87ca060417-signing-key\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.500696 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.500657    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dqkk\" (UniqueName: \"kubernetes.io/projected/04c32f56-70a6-4641-ba00-6c87ca060417-kube-api-access-6dqkk\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.500854 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.500719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:00.500854 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.500762    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:00.500854 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.500778    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/04c32f56-70a6-4641-ba00-6c87ca060417-signing-cabundle\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.500854 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.500809    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/04c32f56-70a6-4641-ba00-6c87ca060417-signing-key\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.501078 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:00.500897    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:24:00.501078 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:00.500921    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:24:00.501078 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:00.500973    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:02.500951178 +0000 UTC m=+64.446690325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : configmap "kube-rbac-proxy" not found
Jul 28 17:24:00.501078 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:00.500990    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:02.500982004 +0000 UTC m=+64.446721146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:24:00.501541 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.501524    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/04c32f56-70a6-4641-ba00-6c87ca060417-signing-cabundle\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.503086 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.503068    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/04c32f56-70a6-4641-ba00-6c87ca060417-signing-key\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.509118 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.509089    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dqkk\" (UniqueName: \"kubernetes.io/projected/04c32f56-70a6-4641-ba00-6c87ca060417-kube-api-access-6dqkk\") pod \"service-ca-658d7b7c78-rwfpq\" (UID: \"04c32f56-70a6-4641-ba00-6c87ca060417\") " pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.646811 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.646729    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq"
Jul 28 17:24:00.774335 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.774305    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rwfpq"]
Jul 28 17:24:00.777742 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:00.777710    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c32f56_70a6_4641_ba00_6c87ca060417.slice/crio-03ff8e5bd4c4bdc5d76576ead9b861b22cbc576355398fcdac8c4ee17922203f WatchSource:0}: Error finding container 03ff8e5bd4c4bdc5d76576ead9b861b22cbc576355398fcdac8c4ee17922203f: Status 404 returned error can't find the container with id 03ff8e5bd4c4bdc5d76576ead9b861b22cbc576355398fcdac8c4ee17922203f
Jul 28 17:24:00.946081 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.946050    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vpgqx_50c96cb9-c926-46eb-8783-993ffbeb9e43/dns-node-resolver/0.log"
Jul 28 17:24:00.991692 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:00.991656    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq" event={"ID":"04c32f56-70a6-4641-ba00-6c87ca060417","Type":"ContainerStarted","Data":"03ff8e5bd4c4bdc5d76576ead9b861b22cbc576355398fcdac8c4ee17922203f"}
Jul 28 17:24:01.945933 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:01.945906    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pq4hj_529a053a-5210-4070-929f-ac5325c9d04b/node-ca/0.log"
Jul 28 17:24:02.017735 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:02.017699    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:24:02.018240 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.017894    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:24:02.018240 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.017955    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls podName:19a545db-ab73-4366-b684-bb8c31012857 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:10.017939409 +0000 UTC m=+71.963678551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fv7pr" (UID: "19a545db-ab73-4366-b684-bb8c31012857") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:24:02.118540 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:02.118503    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:24:02.118736 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:02.118654    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:02.118736 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.118681    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:24:02.118855 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.118755    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls podName:fafd8869-525a-4770-9f01-c5d8bf5ea11a nodeName:}" failed. No retries permitted until 2026-07-28 17:24:10.118738511 +0000 UTC m=+72.064477657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls") pod "cluster-samples-operator-6748489656-cprmc" (UID: "fafd8869-525a-4770-9f01-c5d8bf5ea11a") : secret "samples-operator-tls" not found
Jul 28 17:24:02.118855 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.118773    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:24:02.118855 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.118792    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5996b98fbf-2jsq8: secret "image-registry-tls" not found
Jul 28 17:24:02.118855 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.118849    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls podName:8d7e74f7-415b-4799-9ef3-3019f56ca233 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:10.118832231 +0000 UTC m=+72.064571381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls") pod "image-registry-5996b98fbf-2jsq8" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233") : secret "image-registry-tls" not found
Jul 28 17:24:02.522319 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:02.522277    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:02.522500 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.522421    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:24:02.522500 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:02.522451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:02.522500 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.522500    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:06.52247908 +0000 UTC m=+68.468218249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : configmap "kube-rbac-proxy" not found
Jul 28 17:24:02.522672 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.522570    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:24:02.522672 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:02.522617    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:06.52260239 +0000 UTC m=+68.468341537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:24:02.743134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:02.743045    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgv75_017a2f4a-4f71-435a-9d86-3e2e168787fd/serve-healthcheck-canary/0.log"
Jul 28 17:24:02.999457 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:02.999370    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq" event={"ID":"04c32f56-70a6-4641-ba00-6c87ca060417","Type":"ContainerStarted","Data":"35de5ea4bf016235fd3faf97c5b8c685b7c6b20fbcd6f9c8a9b02a70e4211b3e"}
Jul 28 17:24:03.017797 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:03.017747    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-rwfpq" podStartSLOduration=1.348320804 podStartE2EDuration="3.017731439s" totalImagesPullingTime="1.669410635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:03.016497309 +0000 UTC m=+64.962236485" watchObservedRunningTime="2026-07-28 17:24:03.017731439 +0000 UTC m=+64.963470602"
Jul 28 17:24:04.239603 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.239560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:24:04.242189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.242157    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:24:04.250710 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:04.250683    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 17:24:04.250846 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:04.250760    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs podName:6ae9655b-4180-4a82-9fd7-9eff9b69d437 nodeName:}" failed. No retries permitted until 2026-07-28 17:25:08.250738971 +0000 UTC m=+130.196478119 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs") pod "network-metrics-daemon-nlpg9" (UID: "6ae9655b-4180-4a82-9fd7-9eff9b69d437") : secret "metrics-daemon-secret" not found
Jul 28 17:24:04.340615 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.340573    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:24:04.340803 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.340752    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:24:04.344058 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.344030    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:24:04.344185 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.344071    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:24:04.353386 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.353359    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7-original-pull-secret\") pod \"global-pull-secret-syncer-t7pdd\" (UID: \"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7\") " pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:24:04.353386 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.353347    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:24:04.363712 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.363683    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nww5l\" (UniqueName: \"kubernetes.io/projected/962ce334-76b4-49da-9402-ba293e765556-kube-api-access-nww5l\") pod \"network-check-target-6gtv8\" (UID: \"962ce334-76b4-49da-9402-ba293e765556\") " pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:24:04.506224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.506132    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-27qqv\""
Jul 28 17:24:04.514390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.514361    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:24:04.594217 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.594192    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7pdd"
Jul 28 17:24:04.670263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.670192    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6gtv8"]
Jul 28 17:24:04.673577 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:04.673544    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962ce334_76b4_49da_9402_ba293e765556.slice/crio-eeac8f203a923be5a6c6ae4eac41cbbda5333a6de5e33cfc060f0b5eaf61c86b WatchSource:0}: Error finding container eeac8f203a923be5a6c6ae4eac41cbbda5333a6de5e33cfc060f0b5eaf61c86b: Status 404 returned error can't find the container with id eeac8f203a923be5a6c6ae4eac41cbbda5333a6de5e33cfc060f0b5eaf61c86b
Jul 28 17:24:04.745779 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.745715    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:24:04.745975 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.745803    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:24:04.746211 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.746197    2556 scope.go:117] "RemoveContainer" containerID="10049bd1ba9bdd052cf6f3e3be0ce91db7757cded6f30676390145a690b20d51"
Jul 28 17:24:04.746363 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:04.746345    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-46tzq_openshift-console-operator(d9260f1e-1598-4199-8917-009e5d328ae6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" podUID="d9260f1e-1598-4199-8917-009e5d328ae6"
Jul 28 17:24:04.748421 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.748213    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:24:04.748421 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:04.748338    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:24:04.748421 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:04.748396    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls podName:b63c8dfa-9627-4721-997b-ae04c040ff9a nodeName:}" failed. No retries permitted until 2026-07-28 17:24:36.748377148 +0000 UTC m=+98.694116295 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls") pod "dns-default-b8tjb" (UID: "b63c8dfa-9627-4721-997b-ae04c040ff9a") : secret "dns-default-metrics-tls" not found
Jul 28 17:24:04.749260 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:04.749240    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t7pdd"]
Jul 28 17:24:04.753175 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:04.753151    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35ecc11_bc89_4ff2_a2e0_53cf022b3ac7.slice/crio-e9a67648783b5de6244832652bc6a29acb138a27b91370ebecf546408d7b7f71 WatchSource:0}: Error finding container e9a67648783b5de6244832652bc6a29acb138a27b91370ebecf546408d7b7f71: Status 404 returned error can't find the container with id e9a67648783b5de6244832652bc6a29acb138a27b91370ebecf546408d7b7f71
Jul 28 17:24:05.008375 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:05.008284    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t7pdd" event={"ID":"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7","Type":"ContainerStarted","Data":"e9a67648783b5de6244832652bc6a29acb138a27b91370ebecf546408d7b7f71"}
Jul 28 17:24:05.009423 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:05.009384    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6gtv8" event={"ID":"962ce334-76b4-49da-9402-ba293e765556","Type":"ContainerStarted","Data":"eeac8f203a923be5a6c6ae4eac41cbbda5333a6de5e33cfc060f0b5eaf61c86b"}
Jul 28 17:24:06.565038 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:06.564993    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:06.565437 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:06.565069    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:06.565437 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:06.565168    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:24:06.565437 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:06.565243    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:14.56522316 +0000 UTC m=+76.510962309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : configmap "kube-rbac-proxy" not found
Jul 28 17:24:06.565783 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:06.565752    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:24:06.565928 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:06.565850    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls podName:c364111e-5375-462e-93bd-cbd691f9b2ef nodeName:}" failed. No retries permitted until 2026-07-28 17:24:14.565829616 +0000 UTC m=+76.511568781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k7br8" (UID: "c364111e-5375-462e-93bd-cbd691f9b2ef") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:24:10.030261 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.030222    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t7pdd" event={"ID":"a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7","Type":"ContainerStarted","Data":"3b1e6663f91df385b9a1ca3fa2ffcda095921c098c7f369485f99e3b0818a4c6"}
Jul 28 17:24:10.031772 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.031743    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6gtv8" event={"ID":"962ce334-76b4-49da-9402-ba293e765556","Type":"ContainerStarted","Data":"b94825740ba68af47caccd24a49bee356f723ab17528bf8e24b6f7e226b83ef5"}
Jul 28 17:24:10.031929 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.031902    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:24:10.047818 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.047769    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t7pdd" podStartSLOduration=66.237831379 podStartE2EDuration="1m11.047754342s" totalImagesPullingTime="4.809922963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:10.047414132 +0000 UTC m=+71.993153305" watchObservedRunningTime="2026-07-28 17:24:10.047754342 +0000 UTC m=+71.993493506"
Jul 28 17:24:10.064430 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.064375    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6gtv8" podStartSLOduration=66.179866267 podStartE2EDuration="1m11.064356996s" totalImagesPullingTime="4.884490729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:10.063488338 +0000 UTC m=+72.009227501" watchObservedRunningTime="2026-07-28 17:24:10.064356996 +0000 UTC m=+72.010096160"
Jul 28 17:24:10.100923 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.100860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:24:10.101077 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:10.100985    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:24:10.101077 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:10.101062    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls podName:19a545db-ab73-4366-b684-bb8c31012857 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:26.101047682 +0000 UTC m=+88.046786823 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fv7pr" (UID: "19a545db-ab73-4366-b684-bb8c31012857") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:24:10.201684 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.201646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:10.201684 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.201691    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:24:10.204038 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.204016    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fafd8869-525a-4770-9f01-c5d8bf5ea11a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cprmc\" (UID: \"fafd8869-525a-4770-9f01-c5d8bf5ea11a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:24:10.204150 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.204066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"image-registry-5996b98fbf-2jsq8\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") " pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:10.274442 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.274416    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5vtd5\""
Jul 28 17:24:10.283179 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.283130    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"
Jul 28 17:24:10.362538 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.362512    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7d6vn\""
Jul 28 17:24:10.370509 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.370478    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:10.408083 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.408055    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc"]
Jul 28 17:24:10.506437 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:10.506409    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5996b98fbf-2jsq8"]
Jul 28 17:24:10.509399 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:10.509375    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d7e74f7_415b_4799_9ef3_3019f56ca233.slice/crio-f484cf07eb42e053e01a7479a6cd1460e78f55fac358cf5d0ca250a2d62941b4 WatchSource:0}: Error finding container f484cf07eb42e053e01a7479a6cd1460e78f55fac358cf5d0ca250a2d62941b4: Status 404 returned error can't find the container with id f484cf07eb42e053e01a7479a6cd1460e78f55fac358cf5d0ca250a2d62941b4
Jul 28 17:24:11.036306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:11.036269    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc" event={"ID":"fafd8869-525a-4770-9f01-c5d8bf5ea11a","Type":"ContainerStarted","Data":"302d38cceb38113585a542d0c0a8d6b6b3acce8b2ce6e3b4183681071a5572c1"}
Jul 28 17:24:11.037894 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:11.037853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8" event={"ID":"8d7e74f7-415b-4799-9ef3-3019f56ca233","Type":"ContainerStarted","Data":"9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7"}
Jul 28 17:24:11.038044 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:11.037908    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8" event={"ID":"8d7e74f7-415b-4799-9ef3-3019f56ca233","Type":"ContainerStarted","Data":"f484cf07eb42e053e01a7479a6cd1460e78f55fac358cf5d0ca250a2d62941b4"}
Jul 28 17:24:11.057970 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:11.057915    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8" podStartSLOduration=17.057894902 podStartE2EDuration="17.057894902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:11.057005692 +0000 UTC m=+73.002744855" watchObservedRunningTime="2026-07-28 17:24:11.057894902 +0000 UTC m=+73.003634047"
Jul 28 17:24:12.044109 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:12.044081    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc" event={"ID":"fafd8869-525a-4770-9f01-c5d8bf5ea11a","Type":"ContainerStarted","Data":"c638e1348a346e2ddab06d0e71bbe6f15b320f7cc7b47fae51893432ba3f5598"}
Jul 28 17:24:12.044486 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:12.044313    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:13.051496 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:13.051455    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc" event={"ID":"fafd8869-525a-4770-9f01-c5d8bf5ea11a","Type":"ContainerStarted","Data":"ab3e07866565a5fd4d879f0325e4cd16961bba7a36f179b127ff8aa5005a844d"}
Jul 28 17:24:13.070627 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:13.070577    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cprmc" podStartSLOduration=17.621579109 podStartE2EDuration="19.070562976s" totalImagesPullingTime="1.448983867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:13.069799524 +0000 UTC m=+75.015538703" watchObservedRunningTime="2026-07-28 17:24:13.070562976 +0000 UTC m=+75.016302138"
Jul 28 17:24:14.641531 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:14.641491    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:14.641930 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:14.641548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:14.642158 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:14.642130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c364111e-5375-462e-93bd-cbd691f9b2ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:14.643893 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:14.643845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c364111e-5375-462e-93bd-cbd691f9b2ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k7br8\" (UID: \"c364111e-5375-462e-93bd-cbd691f9b2ef\") " pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:14.697009 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:14.696979    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k7br8"
Jul 28 17:24:14.827001 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:14.826969    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k7br8"]
Jul 28 17:24:14.831272 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:14.831241    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc364111e_5375_462e_93bd_cbd691f9b2ef.slice/crio-cb8142546a22af559e199346a6b4a26f62d1a9b3c1a2df29513c9fd5d56aa754 WatchSource:0}: Error finding container cb8142546a22af559e199346a6b4a26f62d1a9b3c1a2df29513c9fd5d56aa754: Status 404 returned error can't find the container with id cb8142546a22af559e199346a6b4a26f62d1a9b3c1a2df29513c9fd5d56aa754
Jul 28 17:24:15.058139 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:15.058102    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k7br8" event={"ID":"c364111e-5375-462e-93bd-cbd691f9b2ef","Type":"ContainerStarted","Data":"cb8142546a22af559e199346a6b4a26f62d1a9b3c1a2df29513c9fd5d56aa754"}
Jul 28 17:24:19.585318 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:19.585292    2556 scope.go:117] "RemoveContainer" containerID="10049bd1ba9bdd052cf6f3e3be0ce91db7757cded6f30676390145a690b20d51"
Jul 28 17:24:20.078581 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.078548    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:24:20.078989 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.078974    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/1.log"
Jul 28 17:24:20.079048 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.079006    2556 generic.go:356] "Generic (PLEG): container finished" podID="d9260f1e-1598-4199-8917-009e5d328ae6" containerID="08c87afcd6f7b0c1f0a3677b0eb458ff62f54d71fabd4a0c42fea9818467cc73" exitCode=255
Jul 28 17:24:20.079107 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.079085    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" event={"ID":"d9260f1e-1598-4199-8917-009e5d328ae6","Type":"ContainerDied","Data":"08c87afcd6f7b0c1f0a3677b0eb458ff62f54d71fabd4a0c42fea9818467cc73"}
Jul 28 17:24:20.079155 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.079145    2556 scope.go:117] "RemoveContainer" containerID="10049bd1ba9bdd052cf6f3e3be0ce91db7757cded6f30676390145a690b20d51"
Jul 28 17:24:20.079494 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.079475    2556 scope.go:117] "RemoveContainer" containerID="08c87afcd6f7b0c1f0a3677b0eb458ff62f54d71fabd4a0c42fea9818467cc73"
Jul 28 17:24:20.079684 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:20.079656    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-46tzq_openshift-console-operator(d9260f1e-1598-4199-8917-009e5d328ae6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" podUID="d9260f1e-1598-4199-8917-009e5d328ae6"
Jul 28 17:24:20.080588 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.080571    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k7br8" event={"ID":"c364111e-5375-462e-93bd-cbd691f9b2ef","Type":"ContainerStarted","Data":"0fc2d183bce81f704194ee38c23bcdde133baea2f69672d7a0219d32bf8ed9c7"}
Jul 28 17:24:20.624249 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.624213    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"]
Jul 28 17:24:20.627071 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.627047    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:20.631161 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.631131    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 17:24:20.631300 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.631276    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-nb29f\""
Jul 28 17:24:20.631420 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.631401    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 17:24:20.632745 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.632718    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5996b98fbf-2jsq8"]
Jul 28 17:24:20.641295 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.641235    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"]
Jul 28 17:24:20.692968 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.692624    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9c995f4b7-k4vxw"]
Jul 28 17:24:20.695365 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.695020    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.721844 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.721777    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9c995f4b7-k4vxw"]
Jul 28 17:24:20.792497 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792457    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11173307-04ab-4ddc-8ed6-473561ddbd80-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-mf42j\" (UID: \"11173307-04ab-4ddc-8ed6-473561ddbd80\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:20.792685 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792530    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-installation-pull-secrets\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.792685 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792559    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-bound-sa-token\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.792685 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792586    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbnhm\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-kube-api-access-gbnhm\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.792685 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792669    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/11173307-04ab-4ddc-8ed6-473561ddbd80-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-mf42j\" (UID: \"11173307-04ab-4ddc-8ed6-473561ddbd80\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:20.792859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792716    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-registry-tls\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.792859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792759    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-registry-certificates\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.792859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792788    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-ca-trust-extracted\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.792859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792830    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-trusted-ca\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.793040 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.792918    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-image-registry-private-configuration\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-ca-trust-extracted\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894319    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-trusted-ca\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894615 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894376    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-image-registry-private-configuration\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894615 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11173307-04ab-4ddc-8ed6-473561ddbd80-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-mf42j\" (UID: \"11173307-04ab-4ddc-8ed6-473561ddbd80\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:20.894615 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894431    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-installation-pull-secrets\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894615 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894572    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-bound-sa-token\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894615 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbnhm\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-kube-api-access-gbnhm\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894897 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894667    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/11173307-04ab-4ddc-8ed6-473561ddbd80-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-mf42j\" (UID: \"11173307-04ab-4ddc-8ed6-473561ddbd80\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:20.894897 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-registry-tls\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894897 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894723    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-ca-trust-extracted\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.894897 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.894779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-registry-certificates\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.895622 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.895596    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-trusted-ca\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.895727 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.895624    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-registry-certificates\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.896002 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.895975    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/11173307-04ab-4ddc-8ed6-473561ddbd80-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-mf42j\" (UID: \"11173307-04ab-4ddc-8ed6-473561ddbd80\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:20.897432 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.897407    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-installation-pull-secrets\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.897860 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.897804    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11173307-04ab-4ddc-8ed6-473561ddbd80-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-mf42j\" (UID: \"11173307-04ab-4ddc-8ed6-473561ddbd80\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:20.897971 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.897907    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-image-registry-private-configuration\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.898255 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.898234    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-registry-tls\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.913314 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.913286    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbnhm\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-kube-api-access-gbnhm\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.917668 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.917638    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47ff930e-956f-442b-b72b-bb1fc5a9a9ac-bound-sa-token\") pod \"image-registry-9c995f4b7-k4vxw\" (UID: \"47ff930e-956f-442b-b72b-bb1fc5a9a9ac\") " pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:20.940551 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:20.940517    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"
Jul 28 17:24:21.006845 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:21.006814    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:21.087148 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:21.087118    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:24:21.097184 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:21.096923    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-mf42j"]
Jul 28 17:24:21.099688 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:21.099659    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11173307_04ab_4ddc_8ed6_473561ddbd80.slice/crio-f4ff960668e55424c33ecfa30b0cbb1bf721c238df88cef006c91c8ed697ce1e WatchSource:0}: Error finding container f4ff960668e55424c33ecfa30b0cbb1bf721c238df88cef006c91c8ed697ce1e: Status 404 returned error can't find the container with id f4ff960668e55424c33ecfa30b0cbb1bf721c238df88cef006c91c8ed697ce1e
Jul 28 17:24:21.169903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:21.166371    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9c995f4b7-k4vxw"]
Jul 28 17:24:22.093101 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:22.093064    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw" event={"ID":"47ff930e-956f-442b-b72b-bb1fc5a9a9ac","Type":"ContainerStarted","Data":"34a08e0f2d4f274841a39e2ed248aaa376a0b85ed2545a302a1ef25f21941585"}
Jul 28 17:24:22.093578 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:22.093126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw" event={"ID":"47ff930e-956f-442b-b72b-bb1fc5a9a9ac","Type":"ContainerStarted","Data":"1ee342823d0a18b51e3a41bfd10a7f2700a42554558fe95fd22c2039ea5b6d8f"}
Jul 28 17:24:22.093578 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:22.093202    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:22.094482 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:22.094454    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j" event={"ID":"11173307-04ab-4ddc-8ed6-473561ddbd80","Type":"ContainerStarted","Data":"f4ff960668e55424c33ecfa30b0cbb1bf721c238df88cef006c91c8ed697ce1e"}
Jul 28 17:24:22.116046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:22.115989    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw" podStartSLOduration=2.115973554 podStartE2EDuration="2.115973554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:22.114898283 +0000 UTC m=+84.060637458" watchObservedRunningTime="2026-07-28 17:24:22.115973554 +0000 UTC m=+84.061712717"
Jul 28 17:24:24.104312 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:24.104270    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k7br8" event={"ID":"c364111e-5375-462e-93bd-cbd691f9b2ef","Type":"ContainerStarted","Data":"fc63b244499a7426c9358cd12c8da2fe9eb927a356e1128225445146d404b75f"}
Jul 28 17:24:24.106067 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:24.106024    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j" event={"ID":"11173307-04ab-4ddc-8ed6-473561ddbd80","Type":"ContainerStarted","Data":"a3baf63294e21086abaa2dbc36043820b6b928786ed937c0ecfe1d3983e31a37"}
Jul 28 17:24:24.123163 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:24.123106    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-mf42j" podStartSLOduration=1.5917960519999999 podStartE2EDuration="4.1230888s" totalImagesPullingTime="2.531292748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:24.122155988 +0000 UTC m=+86.067895155" watchObservedRunningTime="2026-07-28 17:24:24.1230888 +0000 UTC m=+86.068827965"
Jul 28 17:24:24.746378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:24.746336    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:24:24.746586 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:24.746403    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:24:24.746782 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:24.746766    2556 scope.go:117] "RemoveContainer" containerID="08c87afcd6f7b0c1f0a3677b0eb458ff62f54d71fabd4a0c42fea9818467cc73"
Jul 28 17:24:24.747011 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:24.746986    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-46tzq_openshift-console-operator(d9260f1e-1598-4199-8917-009e5d328ae6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" podUID="d9260f1e-1598-4199-8917-009e5d328ae6"
Jul 28 17:24:26.146751 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:26.146714    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:24:26.149369 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:26.149339    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19a545db-ab73-4366-b684-bb8c31012857-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fv7pr\" (UID: \"19a545db-ab73-4366-b684-bb8c31012857\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:24:26.378452 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:26.378420    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sxcrx\""
Jul 28 17:24:26.386435 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:26.386403    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"
Jul 28 17:24:27.116100 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:27.116073    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr"]
Jul 28 17:24:27.119125 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:27.118989    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a545db_ab73_4366_b684_bb8c31012857.slice/crio-506eb1fa34c48c83662900dff42169fa849909c85ffa731a01945ddf8697a11d WatchSource:0}: Error finding container 506eb1fa34c48c83662900dff42169fa849909c85ffa731a01945ddf8697a11d: Status 404 returned error can't find the container with id 506eb1fa34c48c83662900dff42169fa849909c85ffa731a01945ddf8697a11d
Jul 28 17:24:28.125128 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:28.125065    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr" event={"ID":"19a545db-ab73-4366-b684-bb8c31012857","Type":"ContainerStarted","Data":"506eb1fa34c48c83662900dff42169fa849909c85ffa731a01945ddf8697a11d"}
Jul 28 17:24:28.127941 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:28.127913    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k7br8" event={"ID":"c364111e-5375-462e-93bd-cbd691f9b2ef","Type":"ContainerStarted","Data":"5609f1f1b3ca15d0716e72ef95fb495f57b45c8d5ea981f66373f3a642aa034d"}
Jul 28 17:24:28.149276 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:28.149226    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k7br8" podStartSLOduration=18.05388039 podStartE2EDuration="30.149210954s" totalImagesPullingTime="12.095330564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:23:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:28.147726364 +0000 UTC m=+90.093465530" watchObservedRunningTime="2026-07-28 17:24:28.149210954 +0000 UTC m=+90.094950117"
Jul 28 17:24:29.133308 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:29.133274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr" event={"ID":"19a545db-ab73-4366-b684-bb8c31012857","Type":"ContainerStarted","Data":"832ec03aeaf786df3d057dd08827cfc50f026d15e6e88c98eb109c4c686fb852"}
Jul 28 17:24:29.154423 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:29.153996    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fv7pr" podStartSLOduration=33.232953753 podStartE2EDuration="35.153972129s" totalImagesPullingTime="1.921018376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:29.152118532 +0000 UTC m=+91.097857723" watchObservedRunningTime="2026-07-28 17:24:29.153972129 +0000 UTC m=+91.099711302"
Jul 28 17:24:30.638004 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:30.637967    2556 patch_prober.go:28] interesting pod/image-registry-5996b98fbf-2jsq8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 17:24:30.638474 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:30.638026    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8" podUID="8d7e74f7-415b-4799-9ef3-3019f56ca233" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 17:24:36.824701 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:36.824660    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:24:36.827246 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:36.827215    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b63c8dfa-9627-4721-997b-ae04c040ff9a-metrics-tls\") pod \"dns-default-b8tjb\" (UID: \"b63c8dfa-9627-4721-997b-ae04c040ff9a\") " pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:24:37.063263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.063226    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"]
Jul 28 17:24:37.071185 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.071163    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.080921 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.080726    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 17:24:37.080921 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.080781    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jwt2k\""
Jul 28 17:24:37.080921 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.080786    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 17:24:37.081757 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.081742    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 17:24:37.102555 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.102527    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"]
Jul 28 17:24:37.105619 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.105590    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fqthr\""
Jul 28 17:24:37.114071 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.114040    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:24:37.126881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.126843    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa99f72a-c465-419b-8144-5d91b2742882-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.127020 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.126886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.127020 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.126964    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.127020 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.126991    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swb2w\" (UniqueName: \"kubernetes.io/projected/fa99f72a-c465-419b-8144-5d91b2742882-kube-api-access-swb2w\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.171208 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.171102    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9lhwz"]
Jul 28 17:24:37.205448 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.205419    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.208219 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.208195    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 17:24:37.208511 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.208495    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 17:24:37.209202 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.209176    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 17:24:37.209429 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.209415    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dm6j5\""
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227509    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwf67\" (UniqueName: \"kubernetes.io/projected/9149a65e-6937-4fb3-8571-e3079cf03c44-kube-api-access-nwf67\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227566    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227606    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-wtmp\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227632    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-accelerators-collector-config\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227672    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swb2w\" (UniqueName: \"kubernetes.io/projected/fa99f72a-c465-419b-8144-5d91b2742882-kube-api-access-swb2w\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-root\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227729    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa99f72a-c465-419b-8144-5d91b2742882-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227757    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227781    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-textfile\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227809    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227840    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-tls\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227866    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-sys\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.227923    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9149a65e-6937-4fb3-8571-e3079cf03c44-metrics-client-ca\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:37.228133    2556 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 17:24:37.228321 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:37.228188    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-tls podName:fa99f72a-c465-419b-8144-5d91b2742882 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:37.728168197 +0000 UTC m=+99.673907344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-6zkkv" (UID: "fa99f72a-c465-419b-8144-5d91b2742882") : secret "openshift-state-metrics-tls" not found
Jul 28 17:24:37.229192 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.228979    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa99f72a-c465-419b-8144-5d91b2742882-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.231567 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.231523    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.249623 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.249588    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swb2w\" (UniqueName: \"kubernetes.io/projected/fa99f72a-c465-419b-8144-5d91b2742882-kube-api-access-swb2w\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.299155 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.299119    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b8tjb"]
Jul 28 17:24:37.302929 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:37.302904    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63c8dfa_9627_4721_997b_ae04c040ff9a.slice/crio-d3e9a18e048ffa2a8ead60c052ad00181333a0b9907437021d1ff1d2a1d5dc89 WatchSource:0}: Error finding container d3e9a18e048ffa2a8ead60c052ad00181333a0b9907437021d1ff1d2a1d5dc89: Status 404 returned error can't find the container with id d3e9a18e048ffa2a8ead60c052ad00181333a0b9907437021d1ff1d2a1d5dc89
Jul 28 17:24:37.329178 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329084    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-textfile\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329178 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329137    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329178 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329167    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-tls\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-sys\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329228    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9149a65e-6937-4fb3-8571-e3079cf03c44-metrics-client-ca\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329281    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwf67\" (UniqueName: \"kubernetes.io/projected/9149a65e-6937-4fb3-8571-e3079cf03c44-kube-api-access-nwf67\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-wtmp\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:37.329331    2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329348    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-accelerators-collector-config\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329385    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-root\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:37.329407    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-tls podName:9149a65e-6937-4fb3-8571-e3079cf03c44 nodeName:}" failed. No retries permitted until 2026-07-28 17:24:37.829383836 +0000 UTC m=+99.775122982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-tls") pod "node-exporter-9lhwz" (UID: "9149a65e-6937-4fb3-8571-e3079cf03c44") : secret "node-exporter-tls" not found
Jul 28 17:24:37.329446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329431    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-root\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329467    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-textfile\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329595    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-sys\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.329903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.329711    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-wtmp\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.330208 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.330182    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-accelerators-collector-config\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.330291 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.330209    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9149a65e-6937-4fb3-8571-e3079cf03c44-metrics-client-ca\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.331983 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.331962    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.340708 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.340680    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwf67\" (UniqueName: \"kubernetes.io/projected/9149a65e-6937-4fb3-8571-e3079cf03c44-kube-api-access-nwf67\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.584955 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.584861    2556 scope.go:117] "RemoveContainer" containerID="08c87afcd6f7b0c1f0a3677b0eb458ff62f54d71fabd4a0c42fea9818467cc73"
Jul 28 17:24:37.585081 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:37.585065    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-46tzq_openshift-console-operator(d9260f1e-1598-4199-8917-009e5d328ae6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" podUID="d9260f1e-1598-4199-8917-009e5d328ae6"
Jul 28 17:24:37.733671 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.733639    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.736028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.736008    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa99f72a-c465-419b-8144-5d91b2742882-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6zkkv\" (UID: \"fa99f72a-c465-419b-8144-5d91b2742882\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:37.834811 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.834782    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-tls\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.837179 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.837114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9149a65e-6937-4fb3-8571-e3079cf03c44-node-exporter-tls\") pod \"node-exporter-9lhwz\" (UID: \"9149a65e-6937-4fb3-8571-e3079cf03c44\") " pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:37.982157 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:37.982126    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"
Jul 28 17:24:38.117333 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.117222    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9lhwz"
Jul 28 17:24:38.147271 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:38.147237    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9149a65e_6937_4fb3_8571_e3079cf03c44.slice/crio-0cc60a9c69fa482deb88fb9184b1a71978a50673764fdac70809c62ae0b2d5a7 WatchSource:0}: Error finding container 0cc60a9c69fa482deb88fb9184b1a71978a50673764fdac70809c62ae0b2d5a7: Status 404 returned error can't find the container with id 0cc60a9c69fa482deb88fb9184b1a71978a50673764fdac70809c62ae0b2d5a7
Jul 28 17:24:38.157711 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.157649    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv"]
Jul 28 17:24:38.159888 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:38.159845    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa99f72a_c465_419b_8144_5d91b2742882.slice/crio-a5578381f1631aa708f3a7098e8fd0dd842d670b9f5b156143de622cce11a194 WatchSource:0}: Error finding container a5578381f1631aa708f3a7098e8fd0dd842d670b9f5b156143de622cce11a194: Status 404 returned error can't find the container with id a5578381f1631aa708f3a7098e8fd0dd842d670b9f5b156143de622cce11a194
Jul 28 17:24:38.163451 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.163426    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:24:38.197089 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.197060    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:24:38.197265 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.197128    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9lhwz" event={"ID":"9149a65e-6937-4fb3-8571-e3079cf03c44","Type":"ContainerStarted","Data":"0cc60a9c69fa482deb88fb9184b1a71978a50673764fdac70809c62ae0b2d5a7"}
Jul 28 17:24:38.197265 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.197159    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b8tjb" event={"ID":"b63c8dfa-9627-4721-997b-ae04c040ff9a","Type":"ContainerStarted","Data":"d3e9a18e048ffa2a8ead60c052ad00181333a0b9907437021d1ff1d2a1d5dc89"}
Jul 28 17:24:38.197265 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.197183    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv" event={"ID":"fa99f72a-c465-419b-8144-5d91b2742882","Type":"ContainerStarted","Data":"a5578381f1631aa708f3a7098e8fd0dd842d670b9f5b156143de622cce11a194"}
Jul 28 17:24:38.197510 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.197287    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.199902 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.199855    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 17:24:38.200006 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.199861    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 17:24:38.200887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.200234    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bt97l\""
Jul 28 17:24:38.200887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.200286    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 17:24:38.200887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.200309    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 17:24:38.200887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.200430    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 17:24:38.200887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.200548    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 17:24:38.200887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.200579    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 17:24:38.200887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.200719    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 17:24:38.211005 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.210540    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238763    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-out\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238803    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238822    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-web-config\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238843    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238891    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238909    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238931    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238951    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.238981    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.239000    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.239025    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.239048    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.239702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.239068    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bjqb\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-kube-api-access-2bjqb\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340129 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340092    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340292 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340153    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340292 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340185    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340292 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340292 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340239    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340292 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340281    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340312    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340343    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340370    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340399    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bjqb\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-kube-api-access-2bjqb\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340503    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-out\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340527    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.340552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.340550    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-web-config\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.341757 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.341729    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.342903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.342858    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.344527 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.343887    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-web-config\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.344527 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.344221    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.344743 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.344719    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-out\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.345559 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.345535    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.345651 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.345594    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.345811 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.345788    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.346736 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.346680    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.347548 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.347516    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.348212 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.348148    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.348378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.348332    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.351414 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.351387    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bjqb\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-kube-api-access-2bjqb\") pod \"alertmanager-main-0\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.513842 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.513811    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:24:38.703411 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:38.703324    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:24:38.706676 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:38.706647    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d8c4ea_4473_4d03_8c34_5ab60fb2ce0c.slice/crio-6aa07befc0d5885be83400045cb116a509592ee35a520dcbc44278d4fce1e27f WatchSource:0}: Error finding container 6aa07befc0d5885be83400045cb116a509592ee35a520dcbc44278d4fce1e27f: Status 404 returned error can't find the container with id 6aa07befc0d5885be83400045cb116a509592ee35a520dcbc44278d4fce1e27f
Jul 28 17:24:39.177353 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:39.177312    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv" event={"ID":"fa99f72a-c465-419b-8144-5d91b2742882","Type":"ContainerStarted","Data":"70905943ce693a13f8bc5d1806a443864fae33abad1bc91eae222b77c3120e12"}
Jul 28 17:24:39.177841 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:39.177391    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv" event={"ID":"fa99f72a-c465-419b-8144-5d91b2742882","Type":"ContainerStarted","Data":"0c91319e47e833759f293f29cf373cb44517edcc6ad918f1c31a55bb3a0a901e"}
Jul 28 17:24:39.178763 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:39.178735    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerStarted","Data":"6aa07befc0d5885be83400045cb116a509592ee35a520dcbc44278d4fce1e27f"}
Jul 28 17:24:40.638583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:40.638460    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:41.040477 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.040453    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6gtv8"
Jul 28 17:24:41.188258 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.188226    2556 generic.go:356] "Generic (PLEG): container finished" podID="9149a65e-6937-4fb3-8571-e3079cf03c44" containerID="b51458211d4e03a8e6eecc831801e06c2f1446b817120aaca5bf5125fbe0252b" exitCode=0
Jul 28 17:24:41.188471 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.188302    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9lhwz" event={"ID":"9149a65e-6937-4fb3-8571-e3079cf03c44","Type":"ContainerDied","Data":"b51458211d4e03a8e6eecc831801e06c2f1446b817120aaca5bf5125fbe0252b"}
Jul 28 17:24:41.190188 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.190160    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b8tjb" event={"ID":"b63c8dfa-9627-4721-997b-ae04c040ff9a","Type":"ContainerStarted","Data":"4554621c3b968de2eb13568c2a41e1f540a40093d56d75cf28901c921895e348"}
Jul 28 17:24:41.190301 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.190208    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b8tjb" event={"ID":"b63c8dfa-9627-4721-997b-ae04c040ff9a","Type":"ContainerStarted","Data":"f9b989cc995028f32ba7925d6d8e3b58fa0fc48120961a6834f76cd8e3c83e8d"}
Jul 28 17:24:41.190301 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.190281    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:24:41.192467 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.192444    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv" event={"ID":"fa99f72a-c465-419b-8144-5d91b2742882","Type":"ContainerStarted","Data":"6ecf4a99ec8f46db703f07cc783da7c3790dc0ae863ec436854fafe4296984c9"}
Jul 28 17:24:41.229472 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.229419    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b8tjb" podStartSLOduration=66.66050691 podStartE2EDuration="1m9.229403259s" totalImagesPullingTime="2.568896349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:41.228522249 +0000 UTC m=+103.174261414" watchObservedRunningTime="2026-07-28 17:24:41.229403259 +0000 UTC m=+103.175142476"
Jul 28 17:24:41.248528 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:41.248421    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6zkkv" podStartSLOduration=1.6901003110000001 podStartE2EDuration="4.248403639s" totalImagesPullingTime="2.558303328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:41.247678874 +0000 UTC m=+103.193418037" watchObservedRunningTime="2026-07-28 17:24:41.248403639 +0000 UTC m=+103.194142803"
Jul 28 17:24:42.199352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.199309    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9lhwz" event={"ID":"9149a65e-6937-4fb3-8571-e3079cf03c44","Type":"ContainerStarted","Data":"470204a3c8180cf6698ddb7ed8e14774c8baa18ea1f852223236fde04dcbf723"}
Jul 28 17:24:42.199853 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.199375    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9lhwz" event={"ID":"9149a65e-6937-4fb3-8571-e3079cf03c44","Type":"ContainerStarted","Data":"34a591ffe01b6b9f62d0c74a3a420cd363aa2e64c1ade9c4d036ba4783229fb2"}
Jul 28 17:24:42.200914 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.200887    2556 generic.go:356] "Generic (PLEG): container finished" podID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerID="e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f" exitCode=0
Jul 28 17:24:42.201032 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.200974    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f"}
Jul 28 17:24:42.262316 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.262257    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9lhwz" podStartSLOduration=2.897687612 podStartE2EDuration="5.26223984s" totalImagesPullingTime="2.267715407s" totalInitContainerRuntime="96.836821ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:42.224576722 +0000 UTC m=+104.170315906" watchObservedRunningTime="2026-07-28 17:24:42.26223984 +0000 UTC m=+104.207979004"
Jul 28 17:24:42.443557 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.443520    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-557868565d-5knvj"]
Jul 28 17:24:42.445842 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.445826    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.448221 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.448197    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 17:24:42.448221 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.448213    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 17:24:42.448401 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.448339    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 17:24:42.448401 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.448374    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 17:24:42.448796 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.448781    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5bszj\""
Jul 28 17:24:42.449304 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.449287    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 17:24:42.456986 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.456960    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 17:24:42.462744 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.462717    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-557868565d-5knvj"]
Jul 28 17:24:42.482432 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482390    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-serving-certs-ca-bundle\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.482432 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482432    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-telemeter-trusted-ca-bundle\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.482677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482493    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn6l7\" (UniqueName: \"kubernetes.io/projected/923d8808-1bde-4536-bce2-f1a6d6229d32-kube-api-access-sn6l7\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.482677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-federate-client-tls\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.482677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482566    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-metrics-client-ca\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.482677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482595    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-secret-telemeter-client\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.482677 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.482906 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.482729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-telemeter-client-tls\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.583966 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.583930    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584146 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.583980    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-telemeter-client-tls\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584146 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584008    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-serving-certs-ca-bundle\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584146 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584030    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-telemeter-trusted-ca-bundle\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584146 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584081    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn6l7\" (UniqueName: \"kubernetes.io/projected/923d8808-1bde-4536-bce2-f1a6d6229d32-kube-api-access-sn6l7\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584146 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-federate-client-tls\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584398 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584156    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-metrics-client-ca\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584398 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-secret-telemeter-client\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584984 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584955    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-metrics-client-ca\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.584984 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584975    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-serving-certs-ca-bundle\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.585173 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.584986    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923d8808-1bde-4536-bce2-f1a6d6229d32-telemeter-trusted-ca-bundle\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.586707 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.586686    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-secret-telemeter-client\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.587051 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.587023    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.587126 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.587075    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-telemeter-client-tls\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.587194 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.587176    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/923d8808-1bde-4536-bce2-f1a6d6229d32-federate-client-tls\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.594731 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.594706    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn6l7\" (UniqueName: \"kubernetes.io/projected/923d8808-1bde-4536-bce2-f1a6d6229d32-kube-api-access-sn6l7\") pod \"telemeter-client-557868565d-5knvj\" (UID: \"923d8808-1bde-4536-bce2-f1a6d6229d32\") " pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.759888 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.759792    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-557868565d-5knvj"
Jul 28 17:24:42.923651 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:42.923615    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-557868565d-5knvj"]
Jul 28 17:24:42.927701 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:42.927670    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923d8808_1bde_4536_bce2_f1a6d6229d32.slice/crio-92de5ce55949bb9463fe284db736f47bc266c094aa7d233de90f644ef66f6ccf WatchSource:0}: Error finding container 92de5ce55949bb9463fe284db736f47bc266c094aa7d233de90f644ef66f6ccf: Status 404 returned error can't find the container with id 92de5ce55949bb9463fe284db736f47bc266c094aa7d233de90f644ef66f6ccf
Jul 28 17:24:43.102118 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:43.102048    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9c995f4b7-k4vxw"
Jul 28 17:24:43.206747 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:43.206704    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-557868565d-5knvj" event={"ID":"923d8808-1bde-4536-bce2-f1a6d6229d32","Type":"ContainerStarted","Data":"92de5ce55949bb9463fe284db736f47bc266c094aa7d233de90f644ef66f6ccf"}
Jul 28 17:24:44.216177 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:44.216145    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerStarted","Data":"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"}
Jul 28 17:24:44.216652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:44.216227    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerStarted","Data":"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"}
Jul 28 17:24:44.216652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:44.216266    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerStarted","Data":"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"}
Jul 28 17:24:44.216652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:44.216298    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerStarted","Data":"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"}
Jul 28 17:24:44.216652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:44.216324    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerStarted","Data":"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"}
Jul 28 17:24:45.225293 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:45.225260    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerStarted","Data":"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"}
Jul 28 17:24:45.227005 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:45.226981    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-557868565d-5knvj" event={"ID":"923d8808-1bde-4536-bce2-f1a6d6229d32","Type":"ContainerStarted","Data":"688570973c19b1084b299a6894e0fef6f734dd1428976532764893bfd0520c7e"}
Jul 28 17:24:45.255616 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:45.255566    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.686828025 podStartE2EDuration="7.255548155s" totalImagesPullingTime="4.557534343s" totalInitContainerRuntime="11.185787ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:45.253673027 +0000 UTC m=+107.199412190" watchObservedRunningTime="2026-07-28 17:24:45.255548155 +0000 UTC m=+107.201287318"
Jul 28 17:24:45.662119 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:45.662061    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8" podUID="8d7e74f7-415b-4799-9ef3-3019f56ca233" containerName="registry" containerID="cri-o://9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7" gracePeriod=30
Jul 28 17:24:45.912774 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:45.912703    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:46.016010 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.015980    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d7e74f7-415b-4799-9ef3-3019f56ca233-ca-trust-extracted\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016203 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016053    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-image-registry-private-configuration\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016203 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016088    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxrvj\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-kube-api-access-fxrvj\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016203 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016120    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-bound-sa-token\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016203 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016145    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-certificates\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016203 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016164    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-trusted-ca\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016203 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016198    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-installation-pull-secrets\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016495 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016222    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") pod \"8d7e74f7-415b-4799-9ef3-3019f56ca233\" (UID: \"8d7e74f7-415b-4799-9ef3-3019f56ca233\") "
Jul 28 17:24:46.016617 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016583    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:24:46.016678 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.016629    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:24:46.018673 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.018618    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:24:46.018823 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.018792    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:24:46.019007 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.018986    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:24:46.019074 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.019018    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:24:46.019120 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.019066    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-kube-api-access-fxrvj" (OuterVolumeSpecName: "kube-api-access-fxrvj") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "kube-api-access-fxrvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:24:46.024029 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.024005    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d7e74f7-415b-4799-9ef3-3019f56ca233-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8d7e74f7-415b-4799-9ef3-3019f56ca233" (UID: "8d7e74f7-415b-4799-9ef3-3019f56ca233"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:24:46.117233 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117196    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-image-registry-private-configuration\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.117233 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117227    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxrvj\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-kube-api-access-fxrvj\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.117233 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117237    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-bound-sa-token\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.117442 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117246    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-certificates\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.117442 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117255    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d7e74f7-415b-4799-9ef3-3019f56ca233-trusted-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.117442 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117264    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d7e74f7-415b-4799-9ef3-3019f56ca233-installation-pull-secrets\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.117442 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117272    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d7e74f7-415b-4799-9ef3-3019f56ca233-registry-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.117442 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.117281    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d7e74f7-415b-4799-9ef3-3019f56ca233-ca-trust-extracted\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:24:46.232628 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.232590    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-557868565d-5knvj" event={"ID":"923d8808-1bde-4536-bce2-f1a6d6229d32","Type":"ContainerStarted","Data":"3db327f198c979599d3f476e5342c553c3b3cb364e12d441fea7ef2af0523742"}
Jul 28 17:24:46.233137 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.232653    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-557868565d-5knvj" event={"ID":"923d8808-1bde-4536-bce2-f1a6d6229d32","Type":"ContainerStarted","Data":"0cbedf9695a9ecf0d65a80e23a8a831b3d95e586a860a564befee9542b929ef8"}
Jul 28 17:24:46.233752 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.233725    2556 generic.go:356] "Generic (PLEG): container finished" podID="8d7e74f7-415b-4799-9ef3-3019f56ca233" containerID="9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7" exitCode=0
Jul 28 17:24:46.233862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.233760    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8"
Jul 28 17:24:46.233862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.233809    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8" event={"ID":"8d7e74f7-415b-4799-9ef3-3019f56ca233","Type":"ContainerDied","Data":"9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7"}
Jul 28 17:24:46.233862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.233858    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5996b98fbf-2jsq8" event={"ID":"8d7e74f7-415b-4799-9ef3-3019f56ca233","Type":"ContainerDied","Data":"f484cf07eb42e053e01a7479a6cd1460e78f55fac358cf5d0ca250a2d62941b4"}
Jul 28 17:24:46.233989 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.233894    2556 scope.go:117] "RemoveContainer" containerID="9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7"
Jul 28 17:24:46.242249 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.242231    2556 scope.go:117] "RemoveContainer" containerID="9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7"
Jul 28 17:24:46.242504 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:24:46.242485    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7\": container with ID starting with 9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7 not found: ID does not exist" containerID="9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7"
Jul 28 17:24:46.242548 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.242513    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7"} err="failed to get container status \"9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7\": rpc error: code = NotFound desc = could not find container \"9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7\": container with ID starting with 9c744835ab682034c04d1182c9970b24ea2b63e5a6a8f2dc8fe6a190eda13dd7 not found: ID does not exist"
Jul 28 17:24:46.258991 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.258938    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-557868565d-5knvj" podStartSLOduration=2.050191423 podStartE2EDuration="4.258899201s" totalImagesPullingTime="2.208707778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:46.255337132 +0000 UTC m=+108.201076295" watchObservedRunningTime="2026-07-28 17:24:46.258899201 +0000 UTC m=+108.204638359"
Jul 28 17:24:46.276899 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.276852    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5996b98fbf-2jsq8"]
Jul 28 17:24:46.280917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.280894    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5996b98fbf-2jsq8"]
Jul 28 17:24:46.590789 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:46.590711    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d7e74f7-415b-4799-9ef3-3019f56ca233" path="/var/lib/kubelet/pods/8d7e74f7-415b-4799-9ef3-3019f56ca233/volumes"
Jul 28 17:24:51.203235 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:51.203202    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b8tjb"
Jul 28 17:24:52.584928 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:52.584897    2556 scope.go:117] "RemoveContainer" containerID="08c87afcd6f7b0c1f0a3677b0eb458ff62f54d71fabd4a0c42fea9818467cc73"
Jul 28 17:24:53.265068 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.265042    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:24:53.265230 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.265130    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" event={"ID":"d9260f1e-1598-4199-8917-009e5d328ae6","Type":"ContainerStarted","Data":"a0cfa716e6c960a90a06d3b2ecf20d8599b3b2f696bfc6cb67f7c0e331e95802"}
Jul 28 17:24:53.265430 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.265401    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:24:53.270388 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.270364    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq"
Jul 28 17:24:53.288291 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.288240    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-46tzq" podStartSLOduration=56.535780659 podStartE2EDuration="59.288214088s" totalImagesPullingTime="2.752433429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:53.28714451 +0000 UTC m=+115.232883675" watchObservedRunningTime="2026-07-28 17:24:53.288214088 +0000 UTC m=+115.233953252"
Jul 28 17:24:53.405920 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.405887    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-t97z8"]
Jul 28 17:24:53.406211 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.406199    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d7e74f7-415b-4799-9ef3-3019f56ca233" containerName="registry"
Jul 28 17:24:53.406263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.406212    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d7e74f7-415b-4799-9ef3-3019f56ca233" containerName="registry"
Jul 28 17:24:53.406298 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.406291    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d7e74f7-415b-4799-9ef3-3019f56ca233" containerName="registry"
Jul 28 17:24:53.433920 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.433885    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-t97z8"]
Jul 28 17:24:53.434061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.433972    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-t97z8"
Jul 28 17:24:53.437336 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.437314    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wdf5s\""
Jul 28 17:24:53.438134 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.438118    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 17:24:53.443343 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.443325    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 17:24:53.483781 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.483745    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r694\" (UniqueName: \"kubernetes.io/projected/b2e92fd1-5baf-4c86-9a46-ecbc5a19b013-kube-api-access-2r694\") pod \"downloads-7c97dc4555-t97z8\" (UID: \"b2e92fd1-5baf-4c86-9a46-ecbc5a19b013\") " pod="openshift-console/downloads-7c97dc4555-t97z8"
Jul 28 17:24:53.585130 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.585038    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r694\" (UniqueName: \"kubernetes.io/projected/b2e92fd1-5baf-4c86-9a46-ecbc5a19b013-kube-api-access-2r694\") pod \"downloads-7c97dc4555-t97z8\" (UID: \"b2e92fd1-5baf-4c86-9a46-ecbc5a19b013\") " pod="openshift-console/downloads-7c97dc4555-t97z8"
Jul 28 17:24:53.594539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.594505    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r694\" (UniqueName: \"kubernetes.io/projected/b2e92fd1-5baf-4c86-9a46-ecbc5a19b013-kube-api-access-2r694\") pod \"downloads-7c97dc4555-t97z8\" (UID: \"b2e92fd1-5baf-4c86-9a46-ecbc5a19b013\") " pod="openshift-console/downloads-7c97dc4555-t97z8"
Jul 28 17:24:53.742984 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.742949    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-t97z8"
Jul 28 17:24:53.921578 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:53.921550    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-t97z8"]
Jul 28 17:24:53.924982 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:24:53.924950    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e92fd1_5baf_4c86_9a46_ecbc5a19b013.slice/crio-2af1f740eb26dd1b5295d5a5e27e015a38d327cae7851c99092f97be9d5231ef WatchSource:0}: Error finding container 2af1f740eb26dd1b5295d5a5e27e015a38d327cae7851c99092f97be9d5231ef: Status 404 returned error can't find the container with id 2af1f740eb26dd1b5295d5a5e27e015a38d327cae7851c99092f97be9d5231ef
Jul 28 17:24:54.270023 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:24:54.269986    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-t97z8" event={"ID":"b2e92fd1-5baf-4c86-9a46-ecbc5a19b013","Type":"ContainerStarted","Data":"2af1f740eb26dd1b5295d5a5e27e015a38d327cae7851c99092f97be9d5231ef"}
Jul 28 17:25:04.321763 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.321723    2556 generic.go:356] "Generic (PLEG): container finished" podID="8a3217ae-e76f-4064-bff2-e52f9c649912" containerID="18601024b27a7effe5568d1cfa7b8ae21f9631815a57bf696c6e02902ece95f8" exitCode=0
Jul 28 17:25:04.322207 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.321783    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bgkjm" event={"ID":"8a3217ae-e76f-4064-bff2-e52f9c649912","Type":"ContainerDied","Data":"18601024b27a7effe5568d1cfa7b8ae21f9631815a57bf696c6e02902ece95f8"}
Jul 28 17:25:04.322252 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.322209    2556 scope.go:117] "RemoveContainer" containerID="18601024b27a7effe5568d1cfa7b8ae21f9631815a57bf696c6e02902ece95f8"
Jul 28 17:25:04.749087 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.749053    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-749dd8db85-klh2k"]
Jul 28 17:25:04.751658 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.751629    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.755459 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.755435    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pb8dm\""
Jul 28 17:25:04.755602 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.755493    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 17:25:04.756345 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.756311    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 17:25:04.756482 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.756406    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 17:25:04.757353 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.757311    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 17:25:04.757545 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.757530    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 17:25:04.770488 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.770460    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749dd8db85-klh2k"]
Jul 28 17:25:04.886526 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.886492    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-oauth-config\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.886702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.886538    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-oauth-serving-cert\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.886702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.886572    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-serving-cert\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.886818 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.886724    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q87f\" (UniqueName: \"kubernetes.io/projected/13daa55e-f10d-4394-ac37-f561819b3d6d-kube-api-access-7q87f\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.886818 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.886770    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-console-config\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.886942 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.886824    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-service-ca\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988007 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.987974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q87f\" (UniqueName: \"kubernetes.io/projected/13daa55e-f10d-4394-ac37-f561819b3d6d-kube-api-access-7q87f\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988187 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988018    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-console-config\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988187 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988037    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-service-ca\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988187 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988064    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-oauth-config\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988187 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988096    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-oauth-serving-cert\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988187 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988129    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-serving-cert\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988842 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988814    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-console-config\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988991 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-service-ca\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.988991 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.988924    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-oauth-serving-cert\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.990889 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.990845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-oauth-config\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.990984 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.990889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-serving-cert\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:04.997119 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:04.997098    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q87f\" (UniqueName: \"kubernetes.io/projected/13daa55e-f10d-4394-ac37-f561819b3d6d-kube-api-access-7q87f\") pod \"console-749dd8db85-klh2k\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") " pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:05.062243 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:05.062154    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:08.319138 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:08.319069    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:25:08.321603 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:08.321573    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ae9655b-4180-4a82-9fd7-9eff9b69d437-metrics-certs\") pod \"network-metrics-daemon-nlpg9\" (UID: \"6ae9655b-4180-4a82-9fd7-9eff9b69d437\") " pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:25:08.400065 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:08.400032    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ztptx\""
Jul 28 17:25:08.407917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:08.407886    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nlpg9"
Jul 28 17:25:10.350662 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:10.350608    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bgkjm" event={"ID":"8a3217ae-e76f-4064-bff2-e52f9c649912","Type":"ContainerStarted","Data":"1d09fa2edbf08489ae690db9af6ecf5910c7f6d704df0bd108ca2297e7051ee9"}
Jul 28 17:25:10.403909 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:10.403661    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749dd8db85-klh2k"]
Jul 28 17:25:10.407260 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:25:10.407228    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13daa55e_f10d_4394_ac37_f561819b3d6d.slice/crio-140c4fb9d01bec707cfe3ad9e4d6000be1a94688cd9134d7994d681599b5b0df WatchSource:0}: Error finding container 140c4fb9d01bec707cfe3ad9e4d6000be1a94688cd9134d7994d681599b5b0df: Status 404 returned error can't find the container with id 140c4fb9d01bec707cfe3ad9e4d6000be1a94688cd9134d7994d681599b5b0df
Jul 28 17:25:10.416996 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:10.416973    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nlpg9"]
Jul 28 17:25:10.419363 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:25:10.419322    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae9655b_4180_4a82_9fd7_9eff9b69d437.slice/crio-327cbe2eef519eeb467d658013c55cd5e635d2873c0b04aee9b465c304721910 WatchSource:0}: Error finding container 327cbe2eef519eeb467d658013c55cd5e635d2873c0b04aee9b465c304721910: Status 404 returned error can't find the container with id 327cbe2eef519eeb467d658013c55cd5e635d2873c0b04aee9b465c304721910
Jul 28 17:25:11.362698 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:11.362605    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749dd8db85-klh2k" event={"ID":"13daa55e-f10d-4394-ac37-f561819b3d6d","Type":"ContainerStarted","Data":"140c4fb9d01bec707cfe3ad9e4d6000be1a94688cd9134d7994d681599b5b0df"}
Jul 28 17:25:11.365593 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:11.365539    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-t97z8" event={"ID":"b2e92fd1-5baf-4c86-9a46-ecbc5a19b013","Type":"ContainerStarted","Data":"720dcf977c03bd84d9e138826adcf07e33794af844715112d0685ba680e424a5"}
Jul 28 17:25:11.365977 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:11.365929    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-t97z8"
Jul 28 17:25:11.368660 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:11.368624    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nlpg9" event={"ID":"6ae9655b-4180-4a82-9fd7-9eff9b69d437","Type":"ContainerStarted","Data":"327cbe2eef519eeb467d658013c55cd5e635d2873c0b04aee9b465c304721910"}
Jul 28 17:25:11.386233 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:11.386201    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-t97z8"
Jul 28 17:25:11.397210 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:11.396858    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-t97z8" podStartSLOduration=1.858222691 podStartE2EDuration="18.396833383s" totalImagesPullingTime="16.538610692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:25:11.395852479 +0000 UTC m=+133.341591643" watchObservedRunningTime="2026-07-28 17:25:11.396833383 +0000 UTC m=+133.342572548"
Jul 28 17:25:12.379070 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:12.379032    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nlpg9" event={"ID":"6ae9655b-4180-4a82-9fd7-9eff9b69d437","Type":"ContainerStarted","Data":"760757db5245af6c338beb9725aa1bfd75e144f5f9b2216b0e7e839caa60c877"}
Jul 28 17:25:12.379569 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:12.379097    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nlpg9" event={"ID":"6ae9655b-4180-4a82-9fd7-9eff9b69d437","Type":"ContainerStarted","Data":"61ebd01245b87cbf8b572fbae4f582aea592d513e781609f110dd378bb53c48e"}
Jul 28 17:25:12.405697 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:12.405627    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nlpg9" podStartSLOduration=132.28967201 podStartE2EDuration="2m13.405602919s" totalImagesPullingTime="1.115930909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:25:12.403111688 +0000 UTC m=+134.348850846" watchObservedRunningTime="2026-07-28 17:25:12.405602919 +0000 UTC m=+134.351342085"
Jul 28 17:25:15.398627 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.398586    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749dd8db85-klh2k" event={"ID":"13daa55e-f10d-4394-ac37-f561819b3d6d","Type":"ContainerStarted","Data":"0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478"}
Jul 28 17:25:15.427593 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.427527    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-749dd8db85-klh2k" podStartSLOduration=7.5633936120000005 podStartE2EDuration="11.427507478s" totalImagesPullingTime="3.864113866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:25:15.426390766 +0000 UTC m=+137.372129930" watchObservedRunningTime="2026-07-28 17:25:15.427507478 +0000 UTC m=+137.373246642"
Jul 28 17:25:15.709055 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.709023    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f7fb4d7c8-rgblb"]
Jul 28 17:25:15.741153 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.741109    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7fb4d7c8-rgblb"]
Jul 28 17:25:15.741334 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.741292    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.751783 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.751750    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 17:25:15.794964 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.794920    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-trusted-ca-bundle\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.795148 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.794996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-console-config\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.795148 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.795039    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8l2t\" (UniqueName: \"kubernetes.io/projected/82abd8a7-a453-4d94-b715-280778f82647-kube-api-access-h8l2t\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.795148 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.795127    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-serving-cert\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.795334 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.795156    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-service-ca\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.795334 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.795177    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-oauth-serving-cert\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.795334 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.795204    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-oauth-config\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.895915 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.895863    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-console-config\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.896091 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.895954    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8l2t\" (UniqueName: \"kubernetes.io/projected/82abd8a7-a453-4d94-b715-280778f82647-kube-api-access-h8l2t\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.896091 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.896011    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-serving-cert\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.896091 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.896036    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-service-ca\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.896091 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.896061    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-oauth-serving-cert\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.896091 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.896086    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-oauth-config\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.896348 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.896140    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-trusted-ca-bundle\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.896636 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.896613    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-console-config\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.897013 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.896986    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-oauth-serving-cert\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.897123 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.897112    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-trusted-ca-bundle\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.897494 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.897468    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-service-ca\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.898839 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.898816    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-serving-cert\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.899283 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.899257    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-oauth-config\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:15.904726 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:15.904698    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8l2t\" (UniqueName: \"kubernetes.io/projected/82abd8a7-a453-4d94-b715-280778f82647-kube-api-access-h8l2t\") pod \"console-5f7fb4d7c8-rgblb\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") " pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:16.054655 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:16.054559    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:16.248288 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:16.248257    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7fb4d7c8-rgblb"]
Jul 28 17:25:16.252165 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:25:16.252121    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82abd8a7_a453_4d94_b715_280778f82647.slice/crio-1f2d456b13f9ff4bca84f3879bd769da8964f4fcc65166bdbc1d00f05368c5b6 WatchSource:0}: Error finding container 1f2d456b13f9ff4bca84f3879bd769da8964f4fcc65166bdbc1d00f05368c5b6: Status 404 returned error can't find the container with id 1f2d456b13f9ff4bca84f3879bd769da8964f4fcc65166bdbc1d00f05368c5b6
Jul 28 17:25:16.405089 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:16.405055    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7fb4d7c8-rgblb" event={"ID":"82abd8a7-a453-4d94-b715-280778f82647","Type":"ContainerStarted","Data":"b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca"}
Jul 28 17:25:16.405507 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:16.405114    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7fb4d7c8-rgblb" event={"ID":"82abd8a7-a453-4d94-b715-280778f82647","Type":"ContainerStarted","Data":"1f2d456b13f9ff4bca84f3879bd769da8964f4fcc65166bdbc1d00f05368c5b6"}
Jul 28 17:25:17.435246 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:17.435183    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f7fb4d7c8-rgblb" podStartSLOduration=2.435162683 podStartE2EDuration="2.435162683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:25:17.433073522 +0000 UTC m=+139.378812687" watchObservedRunningTime="2026-07-28 17:25:17.435162683 +0000 UTC m=+139.380901846"
Jul 28 17:25:24.446075 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:24.446044    2556 generic.go:356] "Generic (PLEG): container finished" podID="42149cf6-a57a-4cbb-ae05-81d2bd5ac36e" containerID="36d49e0995e2138cd9f73ea035c612f884aad92e4d84a0a37c100c1631507fa3" exitCode=0
Jul 28 17:25:24.446487 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:24.446117    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4" event={"ID":"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e","Type":"ContainerDied","Data":"36d49e0995e2138cd9f73ea035c612f884aad92e4d84a0a37c100c1631507fa3"}
Jul 28 17:25:24.446487 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:24.446430    2556 scope.go:117] "RemoveContainer" containerID="36d49e0995e2138cd9f73ea035c612f884aad92e4d84a0a37c100c1631507fa3"
Jul 28 17:25:25.062439 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:25.062407    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:25.062624 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:25.062470    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:25.067538 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:25.067514    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:25.451995 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:25.451958    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4" event={"ID":"42149cf6-a57a-4cbb-ae05-81d2bd5ac36e","Type":"ContainerStarted","Data":"5b752dd4c6d1da27bab8848c3c5b4768c0d6e6f4b3671c7664c1c38c34b006ef"}
Jul 28 17:25:25.455817 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:25.455791    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:26.055505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:26.055461    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:26.055684 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:26.055584    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:26.060455 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:26.060431    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:26.459359 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:26.459332    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:25:26.509795 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:26.509761    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749dd8db85-klh2k"]
Jul 28 17:25:52.477723 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.477628    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-749dd8db85-klh2k" podUID="13daa55e-f10d-4394-ac37-f561819b3d6d" containerName="console" containerID="cri-o://0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478" gracePeriod=15
Jul 28 17:25:52.750907 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.750821    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:52.931098 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931058    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-console-config\") pod \"13daa55e-f10d-4394-ac37-f561819b3d6d\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") "
Jul 28 17:25:52.931290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931127    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-serving-cert\") pod \"13daa55e-f10d-4394-ac37-f561819b3d6d\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") "
Jul 28 17:25:52.931290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931171    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q87f\" (UniqueName: \"kubernetes.io/projected/13daa55e-f10d-4394-ac37-f561819b3d6d-kube-api-access-7q87f\") pod \"13daa55e-f10d-4394-ac37-f561819b3d6d\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") "
Jul 28 17:25:52.931290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931198    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-oauth-config\") pod \"13daa55e-f10d-4394-ac37-f561819b3d6d\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") "
Jul 28 17:25:52.931290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931215    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-oauth-serving-cert\") pod \"13daa55e-f10d-4394-ac37-f561819b3d6d\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") "
Jul 28 17:25:52.931290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931251    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-service-ca\") pod \"13daa55e-f10d-4394-ac37-f561819b3d6d\" (UID: \"13daa55e-f10d-4394-ac37-f561819b3d6d\") "
Jul 28 17:25:52.931594 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931563    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-console-config" (OuterVolumeSpecName: "console-config") pod "13daa55e-f10d-4394-ac37-f561819b3d6d" (UID: "13daa55e-f10d-4394-ac37-f561819b3d6d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:25:52.931658 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931614    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "13daa55e-f10d-4394-ac37-f561819b3d6d" (UID: "13daa55e-f10d-4394-ac37-f561819b3d6d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:25:52.931721 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.931656    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-service-ca" (OuterVolumeSpecName: "service-ca") pod "13daa55e-f10d-4394-ac37-f561819b3d6d" (UID: "13daa55e-f10d-4394-ac37-f561819b3d6d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:25:52.933542 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.933518    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "13daa55e-f10d-4394-ac37-f561819b3d6d" (UID: "13daa55e-f10d-4394-ac37-f561819b3d6d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:52.933655 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.933563    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "13daa55e-f10d-4394-ac37-f561819b3d6d" (UID: "13daa55e-f10d-4394-ac37-f561819b3d6d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:52.933655 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:52.933576    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13daa55e-f10d-4394-ac37-f561819b3d6d-kube-api-access-7q87f" (OuterVolumeSpecName: "kube-api-access-7q87f") pod "13daa55e-f10d-4394-ac37-f561819b3d6d" (UID: "13daa55e-f10d-4394-ac37-f561819b3d6d"). InnerVolumeSpecName "kube-api-access-7q87f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:25:53.032263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.032179    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:53.032263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.032210    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:53.032263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.032219    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:53.032263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.032229    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q87f\" (UniqueName: \"kubernetes.io/projected/13daa55e-f10d-4394-ac37-f561819b3d6d-kube-api-access-7q87f\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:53.032263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.032237    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13daa55e-f10d-4394-ac37-f561819b3d6d-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:53.032263 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.032245    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13daa55e-f10d-4394-ac37-f561819b3d6d-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:53.506659 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.506622    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64569bcf6b-7k2qc"]
Jul 28 17:25:53.507172 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.507153    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13daa55e-f10d-4394-ac37-f561819b3d6d" containerName="console"
Jul 28 17:25:53.507241 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.507175    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="13daa55e-f10d-4394-ac37-f561819b3d6d" containerName="console"
Jul 28 17:25:53.507292 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.507276    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="13daa55e-f10d-4394-ac37-f561819b3d6d" containerName="console"
Jul 28 17:25:53.522272 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.522247    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.529732 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.529217    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64569bcf6b-7k2qc"]
Jul 28 17:25:53.562080 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.562044    2556 generic.go:356] "Generic (PLEG): container finished" podID="13daa55e-f10d-4394-ac37-f561819b3d6d" containerID="0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478" exitCode=0
Jul 28 17:25:53.562242 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.562105    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749dd8db85-klh2k"
Jul 28 17:25:53.562242 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.562123    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749dd8db85-klh2k" event={"ID":"13daa55e-f10d-4394-ac37-f561819b3d6d","Type":"ContainerDied","Data":"0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478"}
Jul 28 17:25:53.562242 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.562172    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749dd8db85-klh2k" event={"ID":"13daa55e-f10d-4394-ac37-f561819b3d6d","Type":"ContainerDied","Data":"140c4fb9d01bec707cfe3ad9e4d6000be1a94688cd9134d7994d681599b5b0df"}
Jul 28 17:25:53.562242 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.562195    2556 scope.go:117] "RemoveContainer" containerID="0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478"
Jul 28 17:25:53.570726 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.570708    2556 scope.go:117] "RemoveContainer" containerID="0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478"
Jul 28 17:25:53.571018 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:53.571000    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478\": container with ID starting with 0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478 not found: ID does not exist" containerID="0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478"
Jul 28 17:25:53.571060 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.571028    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478"} err="failed to get container status \"0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478\": rpc error: code = NotFound desc = could not find container \"0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478\": container with ID starting with 0da74b37ff5e3a6c45eeef26446f90d7c10b15b89a4c7b0da8d6501e6a360478 not found: ID does not exist"
Jul 28 17:25:53.584729 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.584707    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749dd8db85-klh2k"]
Jul 28 17:25:53.589315 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.589294    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-749dd8db85-klh2k"]
Jul 28 17:25:53.638112 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.638078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-serving-cert\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.638112 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.638115    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-oauth-serving-cert\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.638302 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.638136    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-trusted-ca-bundle\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.638302 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.638179    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-oauth-config\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.638302 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.638219    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-config\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.638302 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.638271    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-service-ca\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.638425 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.638306    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7tqr\" (UniqueName: \"kubernetes.io/projected/77b07156-b840-43a1-8e5c-3febff7f6bd8-kube-api-access-d7tqr\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.739725 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.739687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-serving-cert\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.739725 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.739732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-oauth-serving-cert\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740026 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.739758    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-trusted-ca-bundle\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740026 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.739785    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-oauth-config\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740026 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.739811    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-config\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740026 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.739849    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-service-ca\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740215 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.740026    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqr\" (UniqueName: \"kubernetes.io/projected/77b07156-b840-43a1-8e5c-3febff7f6bd8-kube-api-access-d7tqr\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740639 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.740617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-service-ca\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740756 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.740666    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-oauth-serving-cert\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740798 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.740758    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-trusted-ca-bundle\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.740798 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.740758    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-config\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.742370 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.742343    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-oauth-config\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.742448 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.742388    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-serving-cert\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.748572 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.748549    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7tqr\" (UniqueName: \"kubernetes.io/projected/77b07156-b840-43a1-8e5c-3febff7f6bd8-kube-api-access-d7tqr\") pod \"console-64569bcf6b-7k2qc\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") " pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.833114 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.833028    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:25:53.995139 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:53.995115    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64569bcf6b-7k2qc"]
Jul 28 17:25:53.997670 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:25:53.997646    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b07156_b840_43a1_8e5c_3febff7f6bd8.slice/crio-1ada6f6c1bc9fe7b20ee665c435f5382e14b7709ebffce0dc66cdf38f31377d8 WatchSource:0}: Error finding container 1ada6f6c1bc9fe7b20ee665c435f5382e14b7709ebffce0dc66cdf38f31377d8: Status 404 returned error can't find the container with id 1ada6f6c1bc9fe7b20ee665c435f5382e14b7709ebffce0dc66cdf38f31377d8
Jul 28 17:25:54.568305 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:54.568270    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64569bcf6b-7k2qc" event={"ID":"77b07156-b840-43a1-8e5c-3febff7f6bd8","Type":"ContainerStarted","Data":"1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614"}
Jul 28 17:25:54.568673 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:54.568321    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64569bcf6b-7k2qc" event={"ID":"77b07156-b840-43a1-8e5c-3febff7f6bd8","Type":"ContainerStarted","Data":"1ada6f6c1bc9fe7b20ee665c435f5382e14b7709ebffce0dc66cdf38f31377d8"}
Jul 28 17:25:54.589967 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:54.589916    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64569bcf6b-7k2qc" podStartSLOduration=1.5899002389999999 podStartE2EDuration="1.589900239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:25:54.587777364 +0000 UTC m=+176.533516556" watchObservedRunningTime="2026-07-28 17:25:54.589900239 +0000 UTC m=+176.535639380"
Jul 28 17:25:54.590448 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:54.590428    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13daa55e-f10d-4394-ac37-f561819b3d6d" path="/var/lib/kubelet/pods/13daa55e-f10d-4394-ac37-f561819b3d6d/volumes"
Jul 28 17:25:57.761636 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:57.761605    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:25:57.762190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:57.761899    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="alertmanager" containerID="cri-o://14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a" gracePeriod=120
Jul 28 17:25:57.762190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:57.761959    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-web" containerID="cri-o://6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b" gracePeriod=120
Jul 28 17:25:57.762190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:57.761964    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="config-reloader" containerID="cri-o://9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e" gracePeriod=120
Jul 28 17:25:57.762190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:57.761983    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-metric" containerID="cri-o://3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd" gracePeriod=120
Jul 28 17:25:57.762190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:57.761973    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy" containerID="cri-o://bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3" gracePeriod=120
Jul 28 17:25:57.762190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:57.761974    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="prom-label-proxy" containerID="cri-o://eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd" gracePeriod=120
Jul 28 17:25:58.590152 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590121    2556 generic.go:356] "Generic (PLEG): container finished" podID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerID="eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd" exitCode=0
Jul 28 17:25:58.590152 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590145    2556 generic.go:356] "Generic (PLEG): container finished" podID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerID="bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3" exitCode=0
Jul 28 17:25:58.590152 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590150    2556 generic.go:356] "Generic (PLEG): container finished" podID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerID="9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e" exitCode=0
Jul 28 17:25:58.590152 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590157    2556 generic.go:356] "Generic (PLEG): container finished" podID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerID="14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a" exitCode=0
Jul 28 17:25:58.590387 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590187    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"}
Jul 28 17:25:58.590387 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590241    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"}
Jul 28 17:25:58.590387 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590266    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"}
Jul 28 17:25:58.590387 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:58.590289    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"}
Jul 28 17:25:59.017417 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.017393    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.089753 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089664    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.089753 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089708    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-volume\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.089753 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089729    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089755    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-cluster-tls-config\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089783    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-trusted-ca-bundle\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089807    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-tls-assets\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089840    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-metrics-client-ca\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089900    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089930    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-web-config\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.089986    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-main-tls\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.090014    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-out\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090424 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.090055    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-main-db\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090424 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.090095    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bjqb\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-kube-api-access-2bjqb\") pod \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\" (UID: \"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c\") "
Jul 28 17:25:59.090424 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.090314    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:25:59.091090 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.090690    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:25:59.091090 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.091025    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:25:59.093806 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.093785    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:59.094384 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.094359    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-kube-api-access-2bjqb" (OuterVolumeSpecName: "kube-api-access-2bjqb") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "kube-api-access-2bjqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:25:59.094479 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.094454    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:25:59.094542 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.094521    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:59.094542 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.094534    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:59.094647 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.094556    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-volume" (OuterVolumeSpecName: "config-volume") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:59.094700 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.094642    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:59.095997 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.095973    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-out" (OuterVolumeSpecName: "config-out") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:25:59.098260 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.098234    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:59.105115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.105090    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-web-config" (OuterVolumeSpecName: "web-config") pod "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" (UID: "58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:59.191266 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191231    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191266 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191261    2556 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-volume\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191266 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191271    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191280    2556 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-cluster-tls-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191289    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191298    2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-tls-assets\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191306    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-metrics-client-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191315    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191324    2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-web-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191332    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-secret-alertmanager-main-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191339    2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-config-out\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191347    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-alertmanager-main-db\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.191490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.191354    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bjqb\" (UniqueName: \"kubernetes.io/projected/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c-kube-api-access-2bjqb\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:25:59.596673 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.596638    2556 generic.go:356] "Generic (PLEG): container finished" podID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerID="3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd" exitCode=0
Jul 28 17:25:59.596673 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.596665    2556 generic.go:356] "Generic (PLEG): container finished" podID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerID="6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b" exitCode=0
Jul 28 17:25:59.596900 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.596705    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.596900 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.596714    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"}
Jul 28 17:25:59.596900 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.596776    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"}
Jul 28 17:25:59.596900 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.596802    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c","Type":"ContainerDied","Data":"6aa07befc0d5885be83400045cb116a509592ee35a520dcbc44278d4fce1e27f"}
Jul 28 17:25:59.596900 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.596835    2556 scope.go:117] "RemoveContainer" containerID="eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"
Jul 28 17:25:59.605307 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.605285    2556 scope.go:117] "RemoveContainer" containerID="3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"
Jul 28 17:25:59.612242 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.612225    2556 scope.go:117] "RemoveContainer" containerID="bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"
Jul 28 17:25:59.618580 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.618564    2556 scope.go:117] "RemoveContainer" containerID="6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"
Jul 28 17:25:59.622417 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.622395    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:25:59.626023 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.625992    2556 scope.go:117] "RemoveContainer" containerID="9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"
Jul 28 17:25:59.632084 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.632039    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:25:59.633066 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.633047    2556 scope.go:117] "RemoveContainer" containerID="14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"
Jul 28 17:25:59.640410 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.640391    2556 scope.go:117] "RemoveContainer" containerID="e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f"
Jul 28 17:25:59.647702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.647679    2556 scope.go:117] "RemoveContainer" containerID="eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"
Jul 28 17:25:59.647959 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:59.647937    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd\": container with ID starting with eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd not found: ID does not exist" containerID="eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"
Jul 28 17:25:59.648023 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.647969    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"} err="failed to get container status \"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd\": rpc error: code = NotFound desc = could not find container \"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd\": container with ID starting with eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd not found: ID does not exist"
Jul 28 17:25:59.648023 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.647988    2556 scope.go:117] "RemoveContainer" containerID="3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"
Jul 28 17:25:59.648239 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:59.648220    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd\": container with ID starting with 3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd not found: ID does not exist" containerID="3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"
Jul 28 17:25:59.648280 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.648245    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"} err="failed to get container status \"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd\": rpc error: code = NotFound desc = could not find container \"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd\": container with ID starting with 3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd not found: ID does not exist"
Jul 28 17:25:59.648280 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.648263    2556 scope.go:117] "RemoveContainer" containerID="bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"
Jul 28 17:25:59.648513 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:59.648490    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3\": container with ID starting with bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3 not found: ID does not exist" containerID="bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"
Jul 28 17:25:59.648583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.648521    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"} err="failed to get container status \"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3\": rpc error: code = NotFound desc = could not find container \"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3\": container with ID starting with bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3 not found: ID does not exist"
Jul 28 17:25:59.648583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.648539    2556 scope.go:117] "RemoveContainer" containerID="6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"
Jul 28 17:25:59.648751 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:59.648733    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b\": container with ID starting with 6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b not found: ID does not exist" containerID="6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"
Jul 28 17:25:59.648812 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.648759    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"} err="failed to get container status \"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b\": rpc error: code = NotFound desc = could not find container \"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b\": container with ID starting with 6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b not found: ID does not exist"
Jul 28 17:25:59.648812 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.648781    2556 scope.go:117] "RemoveContainer" containerID="9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"
Jul 28 17:25:59.649041 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:59.649022    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e\": container with ID starting with 9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e not found: ID does not exist" containerID="9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"
Jul 28 17:25:59.649087 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649049    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"} err="failed to get container status \"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e\": rpc error: code = NotFound desc = could not find container \"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e\": container with ID starting with 9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e not found: ID does not exist"
Jul 28 17:25:59.649087 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649063    2556 scope.go:117] "RemoveContainer" containerID="14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"
Jul 28 17:25:59.649294 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:59.649279    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a\": container with ID starting with 14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a not found: ID does not exist" containerID="14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"
Jul 28 17:25:59.649331 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649298    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"} err="failed to get container status \"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a\": rpc error: code = NotFound desc = could not find container \"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a\": container with ID starting with 14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a not found: ID does not exist"
Jul 28 17:25:59.649331 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649310    2556 scope.go:117] "RemoveContainer" containerID="e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f"
Jul 28 17:25:59.649558 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:25:59.649541    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f\": container with ID starting with e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f not found: ID does not exist" containerID="e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f"
Jul 28 17:25:59.649596 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649562    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f"} err="failed to get container status \"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f\": rpc error: code = NotFound desc = could not find container \"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f\": container with ID starting with e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f not found: ID does not exist"
Jul 28 17:25:59.649596 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649584    2556 scope.go:117] "RemoveContainer" containerID="eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"
Jul 28 17:25:59.649793 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649773    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd"} err="failed to get container status \"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd\": rpc error: code = NotFound desc = could not find container \"eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd\": container with ID starting with eda9b0dd3e9bfe6f830d86b3cdf34e1f244bfd91ec9b27f7003b31f72b1e38bd not found: ID does not exist"
Jul 28 17:25:59.649831 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649794    2556 scope.go:117] "RemoveContainer" containerID="3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"
Jul 28 17:25:59.650016 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.649999    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd"} err="failed to get container status \"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd\": rpc error: code = NotFound desc = could not find container \"3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd\": container with ID starting with 3ea6965b738a0a9617343ba7821e93d509539ec5ed48d91118d843ddcaea2ccd not found: ID does not exist"
Jul 28 17:25:59.650068 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650016    2556 scope.go:117] "RemoveContainer" containerID="bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"
Jul 28 17:25:59.650187 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650170    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3"} err="failed to get container status \"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3\": rpc error: code = NotFound desc = could not find container \"bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3\": container with ID starting with bb1886033df38bb275afe57e4df790fa24c02ccc1e554ec1926df211ab025ed3 not found: ID does not exist"
Jul 28 17:25:59.650227 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650195    2556 scope.go:117] "RemoveContainer" containerID="6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"
Jul 28 17:25:59.650393 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650376    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b"} err="failed to get container status \"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b\": rpc error: code = NotFound desc = could not find container \"6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b\": container with ID starting with 6dc70ede15ca3c523439d4e023ecb3b5d752adb8ab30935baad2fef81f030a3b not found: ID does not exist"
Jul 28 17:25:59.650393 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650393    2556 scope.go:117] "RemoveContainer" containerID="9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"
Jul 28 17:25:59.650601 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650584    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e"} err="failed to get container status \"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e\": rpc error: code = NotFound desc = could not find container \"9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e\": container with ID starting with 9cbb17d9252486194f86337ec86a7360f8c2f3153ba492b35fab3086b8b66c6e not found: ID does not exist"
Jul 28 17:25:59.650639 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650602    2556 scope.go:117] "RemoveContainer" containerID="14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"
Jul 28 17:25:59.650769 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650755    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a"} err="failed to get container status \"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a\": rpc error: code = NotFound desc = could not find container \"14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a\": container with ID starting with 14d2e51d3b4be1683ccd0221dffc2d969729835c8e49b7cca4f05fc9584a3c0a not found: ID does not exist"
Jul 28 17:25:59.650812 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650769    2556 scope.go:117] "RemoveContainer" containerID="e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f"
Jul 28 17:25:59.650954 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.650938    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f"} err="failed to get container status \"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f\": rpc error: code = NotFound desc = could not find container \"e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f\": container with ID starting with e4fd510ccc861766d1c2f1a81774399debeb8f67b23df5bf21e8b280f868fd5f not found: ID does not exist"
Jul 28 17:25:59.684029 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.683995    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:25:59.684313 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684300    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="config-reloader"
Jul 28 17:25:59.684352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684315    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="config-reloader"
Jul 28 17:25:59.684352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684329    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy"
Jul 28 17:25:59.684352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684336    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy"
Jul 28 17:25:59.684352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684347    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-web"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684353    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-web"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684364    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="alertmanager"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684370    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="alertmanager"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684377    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="init-config-reloader"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684383    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="init-config-reloader"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684389    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="prom-label-proxy"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684394    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="prom-label-proxy"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684407    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-metric"
Jul 28 17:25:59.684458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684413    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-metric"
Jul 28 17:25:59.684693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684462    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-metric"
Jul 28 17:25:59.684693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684470    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy"
Jul 28 17:25:59.684693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684476    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="prom-label-proxy"
Jul 28 17:25:59.684693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684482    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="config-reloader"
Jul 28 17:25:59.684693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684487    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="kube-rbac-proxy-web"
Jul 28 17:25:59.684693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.684496    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" containerName="alertmanager"
Jul 28 17:25:59.688164 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.688149    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.695930    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.695966    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696034    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696073    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696104    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ml2b\" (UniqueName: \"kubernetes.io/projected/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-kube-api-access-2ml2b\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696131    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-web-config\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696161    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-config-out\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696229 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696197    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696613 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696237    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696613 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696262    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696613 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696306    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696613 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696334    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696613 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696377    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.696613 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696378    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 17:25:59.696917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696614    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 17:25:59.696917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696645    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 17:25:59.697026 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696926    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 17:25:59.697026 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.696949    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 17:25:59.697200 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.697181    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 17:25:59.697483 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.697467    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bt97l\""
Jul 28 17:25:59.697596 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.697576    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 17:25:59.697690 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.697675    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 17:25:59.706963 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.706936    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 17:25:59.739903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.724994    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:25:59.797736 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.797689    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.797736 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.797737    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.797770    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.797801    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.797825    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ml2b\" (UniqueName: \"kubernetes.io/projected/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-kube-api-access-2ml2b\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.797849    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-web-config\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.797974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-config-out\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798008    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798060    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798124 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798081    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798508 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798211    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798508 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798473    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798508 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798505    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.798652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.798604    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.800545 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.800515    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.800765 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.800742    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.800862 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.800806    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-web-config\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.800964 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.800859    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.800964 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.800892    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.801225 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.801203    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.801293 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.801238    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.801520 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.801503    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.801662 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.801644    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-config-out\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.802636 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.802619    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:25:59.810581 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:25:59.810557    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ml2b\" (UniqueName: \"kubernetes.io/projected/a4e87101-3f3c-40ae-a5d4-894c5f88a0ed-kube-api-access-2ml2b\") pod \"alertmanager-main-0\" (UID: \"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:26:00.000468 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:00.000438    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 17:26:00.187445 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:00.187416    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 17:26:00.200779 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:26:00.200745    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e87101_3f3c_40ae_a5d4_894c5f88a0ed.slice/crio-878976a63ed382741fd561feb4c96e511c65124ae68d1808de0d06efbcfdcbdc WatchSource:0}: Error finding container 878976a63ed382741fd561feb4c96e511c65124ae68d1808de0d06efbcfdcbdc: Status 404 returned error can't find the container with id 878976a63ed382741fd561feb4c96e511c65124ae68d1808de0d06efbcfdcbdc
Jul 28 17:26:00.590734 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:00.590658    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c" path="/var/lib/kubelet/pods/58d8c4ea-4473-4d03-8c34-5ab60fb2ce0c/volumes"
Jul 28 17:26:00.603100 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:00.603068    2556 generic.go:356] "Generic (PLEG): container finished" podID="a4e87101-3f3c-40ae-a5d4-894c5f88a0ed" containerID="55d886f801c010e656fad84f08c91d7fb4745144fe6571ef0827957837d16c65" exitCode=0
Jul 28 17:26:00.603254 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:00.603151    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerDied","Data":"55d886f801c010e656fad84f08c91d7fb4745144fe6571ef0827957837d16c65"}
Jul 28 17:26:00.603254 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:00.603219    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerStarted","Data":"878976a63ed382741fd561feb4c96e511c65124ae68d1808de0d06efbcfdcbdc"}
Jul 28 17:26:01.612934 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:01.612899    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerStarted","Data":"fd994e991f05db4984ef31647c5c085bdd7d9df7be56f15ba0d990c9106f96ed"}
Jul 28 17:26:01.613419 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:01.612981    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerStarted","Data":"6f31eecb5c6ced31b94153282c981cc1fcdb906b9ff30d28577b6d1b42ffd257"}
Jul 28 17:26:01.613419 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:01.613020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerStarted","Data":"be78d2c998d438c43e3e2e4eafbc19f9654431f65561a19125ad7c9f258e8fec"}
Jul 28 17:26:01.613419 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:01.613059    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerStarted","Data":"f7e1b4dd44e148f33cd8fab4da530639eaa6229ceb76d2e20aebf9f55329b09e"}
Jul 28 17:26:01.613419 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:01.613100    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerStarted","Data":"c3c2ab6f480a6ea828c1c0affafacbe5c6fba0940467d16dbd2c5182a0456d07"}
Jul 28 17:26:01.613419 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:01.613133    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4e87101-3f3c-40ae-a5d4-894c5f88a0ed","Type":"ContainerStarted","Data":"200f2d7a0d40f568f7b7df2e8a74d664f757ad7377449b83cd75f668e1ba02ae"}
Jul 28 17:26:01.644837 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:01.644777    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.629560178 podStartE2EDuration="2.6447633s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.203122ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:26:01.643822556 +0000 UTC m=+183.589561755" watchObservedRunningTime="2026-07-28 17:26:01.6447633 +0000 UTC m=+183.590502461"
Jul 28 17:26:03.834061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:03.834029    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:26:03.834061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:03.834082    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:26:03.838986 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:03.838963    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:26:04.628992 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:04.628963    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:26:04.678663 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:04.678628    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7fb4d7c8-rgblb"]
Jul 28 17:26:29.697903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:29.697815    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f7fb4d7c8-rgblb" podUID="82abd8a7-a453-4d94-b715-280778f82647" containerName="console" containerID="cri-o://b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca" gracePeriod=15
Jul 28 17:26:29.948221 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:29.948152    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:26:30.073965 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.073927    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-trusted-ca-bundle\") pod \"82abd8a7-a453-4d94-b715-280778f82647\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") "
Jul 28 17:26:30.074154 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.073979    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-serving-cert\") pod \"82abd8a7-a453-4d94-b715-280778f82647\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") "
Jul 28 17:26:30.074154 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.073999    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-console-config\") pod \"82abd8a7-a453-4d94-b715-280778f82647\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") "
Jul 28 17:26:30.074154 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074061    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-oauth-serving-cert\") pod \"82abd8a7-a453-4d94-b715-280778f82647\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") "
Jul 28 17:26:30.074154 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074084    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8l2t\" (UniqueName: \"kubernetes.io/projected/82abd8a7-a453-4d94-b715-280778f82647-kube-api-access-h8l2t\") pod \"82abd8a7-a453-4d94-b715-280778f82647\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") "
Jul 28 17:26:30.074154 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074104    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-oauth-config\") pod \"82abd8a7-a453-4d94-b715-280778f82647\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") "
Jul 28 17:26:30.074154 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074128    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-service-ca\") pod \"82abd8a7-a453-4d94-b715-280778f82647\" (UID: \"82abd8a7-a453-4d94-b715-280778f82647\") "
Jul 28 17:26:30.074531 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074503    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-console-config" (OuterVolumeSpecName: "console-config") pod "82abd8a7-a453-4d94-b715-280778f82647" (UID: "82abd8a7-a453-4d94-b715-280778f82647"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:26:30.074531 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074514    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-service-ca" (OuterVolumeSpecName: "service-ca") pod "82abd8a7-a453-4d94-b715-280778f82647" (UID: "82abd8a7-a453-4d94-b715-280778f82647"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:26:30.074794 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074497    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "82abd8a7-a453-4d94-b715-280778f82647" (UID: "82abd8a7-a453-4d94-b715-280778f82647"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:26:30.074794 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.074743    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "82abd8a7-a453-4d94-b715-280778f82647" (UID: "82abd8a7-a453-4d94-b715-280778f82647"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:26:30.076286 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.076265    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "82abd8a7-a453-4d94-b715-280778f82647" (UID: "82abd8a7-a453-4d94-b715-280778f82647"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:26:30.076566 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.076544    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "82abd8a7-a453-4d94-b715-280778f82647" (UID: "82abd8a7-a453-4d94-b715-280778f82647"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:26:30.076566 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.076544    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82abd8a7-a453-4d94-b715-280778f82647-kube-api-access-h8l2t" (OuterVolumeSpecName: "kube-api-access-h8l2t") pod "82abd8a7-a453-4d94-b715-280778f82647" (UID: "82abd8a7-a453-4d94-b715-280778f82647"). InnerVolumeSpecName "kube-api-access-h8l2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:26:30.175583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.175545    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:26:30.175583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.175579    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:26:30.175583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.175591    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:26:30.175823 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.175603    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8l2t\" (UniqueName: \"kubernetes.io/projected/82abd8a7-a453-4d94-b715-280778f82647-kube-api-access-h8l2t\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:26:30.175823 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.175618    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82abd8a7-a453-4d94-b715-280778f82647-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:26:30.175823 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.175628    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:26:30.175823 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.175639    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82abd8a7-a453-4d94-b715-280778f82647-trusted-ca-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:26:30.729131 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.729094    2556 generic.go:356] "Generic (PLEG): container finished" podID="82abd8a7-a453-4d94-b715-280778f82647" containerID="b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca" exitCode=0
Jul 28 17:26:30.729626 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.729140    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7fb4d7c8-rgblb"
Jul 28 17:26:30.729626 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.729181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7fb4d7c8-rgblb" event={"ID":"82abd8a7-a453-4d94-b715-280778f82647","Type":"ContainerDied","Data":"b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca"}
Jul 28 17:26:30.729626 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.729245    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7fb4d7c8-rgblb" event={"ID":"82abd8a7-a453-4d94-b715-280778f82647","Type":"ContainerDied","Data":"1f2d456b13f9ff4bca84f3879bd769da8964f4fcc65166bdbc1d00f05368c5b6"}
Jul 28 17:26:30.729626 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.729278    2556 scope.go:117] "RemoveContainer" containerID="b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca"
Jul 28 17:26:30.737706 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.737687    2556 scope.go:117] "RemoveContainer" containerID="b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca"
Jul 28 17:26:30.738037 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:26:30.738018    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca\": container with ID starting with b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca not found: ID does not exist" containerID="b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca"
Jul 28 17:26:30.738109 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.738048    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca"} err="failed to get container status \"b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca\": rpc error: code = NotFound desc = could not find container \"b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca\": container with ID starting with b7c910a58b2572b42ec87fb99d6b7ce827d4d126955141c20f8e11bcab5cd4ca not found: ID does not exist"
Jul 28 17:26:30.745622 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.745593    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7fb4d7c8-rgblb"]
Jul 28 17:26:30.750163 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:30.750138    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f7fb4d7c8-rgblb"]
Jul 28 17:26:32.589480 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:26:32.589447    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82abd8a7-a453-4d94-b715-280778f82647" path="/var/lib/kubelet/pods/82abd8a7-a453-4d94-b715-280778f82647/volumes"
Jul 28 17:27:12.437750 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.437711    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"]
Jul 28 17:27:12.438379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.438222    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82abd8a7-a453-4d94-b715-280778f82647" containerName="console"
Jul 28 17:27:12.438379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.438242    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="82abd8a7-a453-4d94-b715-280778f82647" containerName="console"
Jul 28 17:27:12.438379 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.438326    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="82abd8a7-a453-4d94-b715-280778f82647" containerName="console"
Jul 28 17:27:12.441407 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.441386    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.444143 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.444120    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:27:12.444256 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.444120    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5pqwh\""
Jul 28 17:27:12.444894 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.444862    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:27:12.454360 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.454337    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"]
Jul 28 17:27:12.538774 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.538742    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.538975 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.538804    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.538975 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.538894    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6cqw\" (UniqueName: \"kubernetes.io/projected/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-kube-api-access-n6cqw\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.639368 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.639329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.639552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.639388    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.639552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.639415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6cqw\" (UniqueName: \"kubernetes.io/projected/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-kube-api-access-n6cqw\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.639740 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.639720    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.639788 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.639723    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.648520 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.648493    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6cqw\" (UniqueName: \"kubernetes.io/projected/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-kube-api-access-n6cqw\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.750259 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.750175    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:12.879174 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.879146    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"]
Jul 28 17:27:12.882930 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:27:12.882904    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29f7f17_61ef_45a1_af5c_5d64d1f6796d.slice/crio-d38f4326242895aa3c25578cea9eae0d9aec014742062b0b67299577ba60599a WatchSource:0}: Error finding container d38f4326242895aa3c25578cea9eae0d9aec014742062b0b67299577ba60599a: Status 404 returned error can't find the container with id d38f4326242895aa3c25578cea9eae0d9aec014742062b0b67299577ba60599a
Jul 28 17:27:12.891679 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:12.891652    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc" event={"ID":"d29f7f17-61ef-45a1-af5c-5d64d1f6796d","Type":"ContainerStarted","Data":"d38f4326242895aa3c25578cea9eae0d9aec014742062b0b67299577ba60599a"}
Jul 28 17:27:19.924408 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:19.924323    2556 generic.go:356] "Generic (PLEG): container finished" podID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerID="d8a0c186de00849ec09829d6ac1584af6c53d60211c2743c0e26ba13a2ee0546" exitCode=0
Jul 28 17:27:19.924408 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:19.924364    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc" event={"ID":"d29f7f17-61ef-45a1-af5c-5d64d1f6796d","Type":"ContainerDied","Data":"d8a0c186de00849ec09829d6ac1584af6c53d60211c2743c0e26ba13a2ee0546"}
Jul 28 17:27:22.937538 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:22.937498    2556 generic.go:356] "Generic (PLEG): container finished" podID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerID="5a2c9796a3a2a4fcbac2c66a8753516ae1e431b56da2a8e9112df50a2b1e11dc" exitCode=0
Jul 28 17:27:22.937934 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:22.937558    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc" event={"ID":"d29f7f17-61ef-45a1-af5c-5d64d1f6796d","Type":"ContainerDied","Data":"5a2c9796a3a2a4fcbac2c66a8753516ae1e431b56da2a8e9112df50a2b1e11dc"}
Jul 28 17:27:30.980186 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:30.980151    2556 generic.go:356] "Generic (PLEG): container finished" podID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerID="11ad4dd1e3da0983616f36eb0b32c0599af1f5ba5270ce4ec9cdaa729db7d3b2" exitCode=0
Jul 28 17:27:30.980565 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:30.980235    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc" event={"ID":"d29f7f17-61ef-45a1-af5c-5d64d1f6796d","Type":"ContainerDied","Data":"11ad4dd1e3da0983616f36eb0b32c0599af1f5ba5270ce4ec9cdaa729db7d3b2"}
Jul 28 17:27:32.104215 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.104189    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:32.208944 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.208903    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-util\") pod \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") "
Jul 28 17:27:32.208944 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.208944    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-bundle\") pod \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") "
Jul 28 17:27:32.209203 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.208987    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6cqw\" (UniqueName: \"kubernetes.io/projected/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-kube-api-access-n6cqw\") pod \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\" (UID: \"d29f7f17-61ef-45a1-af5c-5d64d1f6796d\") "
Jul 28 17:27:32.209611 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.209589    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-bundle" (OuterVolumeSpecName: "bundle") pod "d29f7f17-61ef-45a1-af5c-5d64d1f6796d" (UID: "d29f7f17-61ef-45a1-af5c-5d64d1f6796d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:27:32.211213 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.211189    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-kube-api-access-n6cqw" (OuterVolumeSpecName: "kube-api-access-n6cqw") pod "d29f7f17-61ef-45a1-af5c-5d64d1f6796d" (UID: "d29f7f17-61ef-45a1-af5c-5d64d1f6796d"). InnerVolumeSpecName "kube-api-access-n6cqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:27:32.213070 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.213044    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-util" (OuterVolumeSpecName: "util") pod "d29f7f17-61ef-45a1-af5c-5d64d1f6796d" (UID: "d29f7f17-61ef-45a1-af5c-5d64d1f6796d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:27:32.310133 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.310040    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-util\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:27:32.310133 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.310071    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:27:32.310133 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.310084    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6cqw\" (UniqueName: \"kubernetes.io/projected/d29f7f17-61ef-45a1-af5c-5d64d1f6796d-kube-api-access-n6cqw\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:27:32.988861 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.988834    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc"
Jul 28 17:27:32.989032 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.988852    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5pgtcc" event={"ID":"d29f7f17-61ef-45a1-af5c-5d64d1f6796d","Type":"ContainerDied","Data":"d38f4326242895aa3c25578cea9eae0d9aec014742062b0b67299577ba60599a"}
Jul 28 17:27:32.989032 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:32.988917    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d38f4326242895aa3c25578cea9eae0d9aec014742062b0b67299577ba60599a"
Jul 28 17:27:35.583909 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.583877    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"]
Jul 28 17:27:35.584296 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.584200    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerName="util"
Jul 28 17:27:35.584296 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.584211    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerName="util"
Jul 28 17:27:35.584296 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.584227    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerName="pull"
Jul 28 17:27:35.584296 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.584235    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerName="pull"
Jul 28 17:27:35.584296 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.584244    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerName="extract"
Jul 28 17:27:35.584296 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.584249    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerName="extract"
Jul 28 17:27:35.584469 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.584312    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d29f7f17-61ef-45a1-af5c-5d64d1f6796d" containerName="extract"
Jul 28 17:27:35.627322 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.627285    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"]
Jul 28 17:27:35.627476 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.627397    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:35.637098 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.637080    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 28 17:27:35.638569 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.638552    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-hnm4s\""
Jul 28 17:27:35.638638 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.638625    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 28 17:27:35.638841 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.638826    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 28 17:27:35.738293 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.738260    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zkdc\" (UniqueName: \"kubernetes.io/projected/32343414-91c3-409a-950c-ce777e0b5dcf-kube-api-access-6zkdc\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-wchs9\" (UID: \"32343414-91c3-409a-950c-ce777e0b5dcf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:35.738453 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.738322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/32343414-91c3-409a-950c-ce777e0b5dcf-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-wchs9\" (UID: \"32343414-91c3-409a-950c-ce777e0b5dcf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:35.839712 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.839635    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/32343414-91c3-409a-950c-ce777e0b5dcf-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-wchs9\" (UID: \"32343414-91c3-409a-950c-ce777e0b5dcf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:35.839712 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.839695    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zkdc\" (UniqueName: \"kubernetes.io/projected/32343414-91c3-409a-950c-ce777e0b5dcf-kube-api-access-6zkdc\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-wchs9\" (UID: \"32343414-91c3-409a-950c-ce777e0b5dcf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:35.842008 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.841979    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/32343414-91c3-409a-950c-ce777e0b5dcf-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-wchs9\" (UID: \"32343414-91c3-409a-950c-ce777e0b5dcf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:35.849895 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.849851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zkdc\" (UniqueName: \"kubernetes.io/projected/32343414-91c3-409a-950c-ce777e0b5dcf-kube-api-access-6zkdc\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-wchs9\" (UID: \"32343414-91c3-409a-950c-ce777e0b5dcf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:35.937178 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:35.937149    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:36.090719 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:36.090694    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"]
Jul 28 17:27:36.093189 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:27:36.093166    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32343414_91c3_409a_950c_ce777e0b5dcf.slice/crio-0571532cb885beb28ba3793f0ffd775d0074d2a2f8020b9e8ae235b9db3cb022 WatchSource:0}: Error finding container 0571532cb885beb28ba3793f0ffd775d0074d2a2f8020b9e8ae235b9db3cb022: Status 404 returned error can't find the container with id 0571532cb885beb28ba3793f0ffd775d0074d2a2f8020b9e8ae235b9db3cb022
Jul 28 17:27:37.012546 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:37.012504    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9" event={"ID":"32343414-91c3-409a-950c-ce777e0b5dcf","Type":"ContainerStarted","Data":"0571532cb885beb28ba3793f0ffd775d0074d2a2f8020b9e8ae235b9db3cb022"}
Jul 28 17:27:40.034616 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.034576    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9" event={"ID":"32343414-91c3-409a-950c-ce777e0b5dcf","Type":"ContainerStarted","Data":"58544c0a72ce06dc10d5c3237e0b247b7763d3f5f479bf69ede2cee3e84a2927"}
Jul 28 17:27:40.035071 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.034658    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:27:40.063300 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.063247    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9" podStartSLOduration=1.757287327 podStartE2EDuration="5.063230586s" totalImagesPullingTime="3.305943259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:40.062711311 +0000 UTC m=+282.008450475" watchObservedRunningTime="2026-07-28 17:27:40.063230586 +0000 UTC m=+282.008969749"
Jul 28 17:27:40.283136 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.283102    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-xrfmj"]
Jul 28 17:27:40.298456 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.298371    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.299898 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.299860    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-xrfmj"]
Jul 28 17:27:40.301056 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.301034    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 28 17:27:40.301463 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.301126    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-djwvk\""
Jul 28 17:27:40.301463 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.301309    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 28 17:27:40.380846 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.380817    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/f45c022b-588c-474d-8d6a-ac6a6b340bda-cabundle0\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.381039 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.380943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmrj5\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-kube-api-access-cmrj5\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.381039 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.381001    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.482408 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.482367    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmrj5\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-kube-api-access-cmrj5\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.482605 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.482426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.482605 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.482471    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/f45c022b-588c-474d-8d6a-ac6a6b340bda-cabundle0\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.482605 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.482562    2556 secret.go:281] references non-existent secret key: ca.crt
Jul 28 17:27:40.482605 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.482590    2556 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 28 17:27:40.482605 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.482601    2556 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-xrfmj: references non-existent secret key: ca.crt
Jul 28 17:27:40.482864 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.482675    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates podName:f45c022b-588c-474d-8d6a-ac6a6b340bda nodeName:}" failed. No retries permitted until 2026-07-28 17:27:40.982654746 +0000 UTC m=+282.928393906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates") pod "keda-operator-77f648d694-xrfmj" (UID: "f45c022b-588c-474d-8d6a-ac6a6b340bda") : references non-existent secret key: ca.crt
Jul 28 17:27:40.483217 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.483195    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/f45c022b-588c-474d-8d6a-ac6a6b340bda-cabundle0\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.494635 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.494612    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmrj5\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-kube-api-access-cmrj5\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.834221 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.834189    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"]
Jul 28 17:27:40.836343 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.836327    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:40.838521 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.838503    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 28 17:27:40.850003 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.849972    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"]
Jul 28 17:27:40.886829 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.886794    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/5424049e-4392-4be0-9016-cd08e91627d5-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:40.887023 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.886955    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-certificates\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:40.887077 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.887016    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj94g\" (UniqueName: \"kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-kube-api-access-cj94g\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:40.987575 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.987535    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-certificates\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:40.987767 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.987599    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj94g\" (UniqueName: \"kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-kube-api-access-cj94g\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:40.987767 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.987647    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/5424049e-4392-4be0-9016-cd08e91627d5-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:40.987767 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987670    2556 secret.go:281] references non-existent secret key: tls.crt
Jul 28 17:27:40.987767 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987690    2556 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 28 17:27:40.987767 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987710    2556 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-42dx7: references non-existent secret key: tls.crt
Jul 28 17:27:40.987767 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987765    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-certificates podName:5424049e-4392-4be0-9016-cd08e91627d5 nodeName:}" failed. No retries permitted until 2026-07-28 17:27:41.487744514 +0000 UTC m=+283.433483658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-certificates") pod "keda-metrics-apiserver-55557df8c-42dx7" (UID: "5424049e-4392-4be0-9016-cd08e91627d5") : references non-existent secret key: tls.crt
Jul 28 17:27:40.988138 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.987782    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:40.988138 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987886    2556 secret.go:281] references non-existent secret key: ca.crt
Jul 28 17:27:40.988138 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987900    2556 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 28 17:27:40.988138 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987910    2556 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-xrfmj: references non-existent secret key: ca.crt
Jul 28 17:27:40.988138 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:40.987944    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates podName:f45c022b-588c-474d-8d6a-ac6a6b340bda nodeName:}" failed. No retries permitted until 2026-07-28 17:27:41.987932848 +0000 UTC m=+283.933671994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates") pod "keda-operator-77f648d694-xrfmj" (UID: "f45c022b-588c-474d-8d6a-ac6a6b340bda") : references non-existent secret key: ca.crt
Jul 28 17:27:40.988138 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:40.988033    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/5424049e-4392-4be0-9016-cd08e91627d5-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:41.001564 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.001538    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj94g\" (UniqueName: \"kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-kube-api-access-cj94g\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:41.449658 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.449627    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-79sxg"]
Jul 28 17:27:41.452256 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.452236    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:41.454547 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.454525    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 28 17:27:41.464988 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.464960    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-79sxg"]
Jul 28 17:27:41.492920 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.492865    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-certificates\") pod \"keda-admission-56f9798bb4-79sxg\" (UID: \"0487363e-1875-4e62-9289-141ccb5d0529\") " pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:41.493085 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.493036    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdn7m\" (UniqueName: \"kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-kube-api-access-sdn7m\") pod \"keda-admission-56f9798bb4-79sxg\" (UID: \"0487363e-1875-4e62-9289-141ccb5d0529\") " pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:41.493137 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.493073    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-certificates\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:41.495687 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.495664    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5424049e-4392-4be0-9016-cd08e91627d5-certificates\") pod \"keda-metrics-apiserver-55557df8c-42dx7\" (UID: \"5424049e-4392-4be0-9016-cd08e91627d5\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:41.593914 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.593859    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdn7m\" (UniqueName: \"kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-kube-api-access-sdn7m\") pod \"keda-admission-56f9798bb4-79sxg\" (UID: \"0487363e-1875-4e62-9289-141ccb5d0529\") " pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:41.594109 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.593958    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-certificates\") pod \"keda-admission-56f9798bb4-79sxg\" (UID: \"0487363e-1875-4e62-9289-141ccb5d0529\") " pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:41.594109 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:41.594064    2556 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 28 17:27:41.594109 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:41.594089    2556 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-79sxg: secret "keda-admission-webhooks-certs" not found
Jul 28 17:27:41.594310 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:27:41.594150    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-certificates podName:0487363e-1875-4e62-9289-141ccb5d0529 nodeName:}" failed. No retries permitted until 2026-07-28 17:27:42.09413089 +0000 UTC m=+284.039870044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-certificates") pod "keda-admission-56f9798bb4-79sxg" (UID: "0487363e-1875-4e62-9289-141ccb5d0529") : secret "keda-admission-webhooks-certs" not found
Jul 28 17:27:41.608738 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.608704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdn7m\" (UniqueName: \"kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-kube-api-access-sdn7m\") pod \"keda-admission-56f9798bb4-79sxg\" (UID: \"0487363e-1875-4e62-9289-141ccb5d0529\") " pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:41.746076 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.745993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:41.905220 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.905198    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"]
Jul 28 17:27:41.907766 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:27:41.907738    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5424049e_4392_4be0_9016_cd08e91627d5.slice/crio-3c66f82ff28cd913a433f9b0cd908aaa54f629fce3b988f660e66ccd215f3129 WatchSource:0}: Error finding container 3c66f82ff28cd913a433f9b0cd908aaa54f629fce3b988f660e66ccd215f3129: Status 404 returned error can't find the container with id 3c66f82ff28cd913a433f9b0cd908aaa54f629fce3b988f660e66ccd215f3129
Jul 28 17:27:41.998681 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:41.998595    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:42.000939 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.000909    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f45c022b-588c-474d-8d6a-ac6a6b340bda-certificates\") pod \"keda-operator-77f648d694-xrfmj\" (UID: \"f45c022b-588c-474d-8d6a-ac6a6b340bda\") " pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:42.043945 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.043904    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7" event={"ID":"5424049e-4392-4be0-9016-cd08e91627d5","Type":"ContainerStarted","Data":"3c66f82ff28cd913a433f9b0cd908aaa54f629fce3b988f660e66ccd215f3129"}
Jul 28 17:27:42.099657 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.099618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-certificates\") pod \"keda-admission-56f9798bb4-79sxg\" (UID: \"0487363e-1875-4e62-9289-141ccb5d0529\") " pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:42.102002 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.101979    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0487363e-1875-4e62-9289-141ccb5d0529-certificates\") pod \"keda-admission-56f9798bb4-79sxg\" (UID: \"0487363e-1875-4e62-9289-141ccb5d0529\") " pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:42.111877 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.111840    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:42.261382 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.261358    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-xrfmj"]
Jul 28 17:27:42.263670 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:27:42.263641    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45c022b_588c_474d_8d6a_ac6a6b340bda.slice/crio-8fbf6ebd965c1a056f72b23212ef876cb9c13e04ea5d7572cacb7147df02d7b6 WatchSource:0}: Error finding container 8fbf6ebd965c1a056f72b23212ef876cb9c13e04ea5d7572cacb7147df02d7b6: Status 404 returned error can't find the container with id 8fbf6ebd965c1a056f72b23212ef876cb9c13e04ea5d7572cacb7147df02d7b6
Jul 28 17:27:42.364069 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.364028    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:42.499079 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:42.499054    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-79sxg"]
Jul 28 17:27:42.501604 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:27:42.501578    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0487363e_1875_4e62_9289_141ccb5d0529.slice/crio-8076251e342f56c32051dade740deb9ad535be99e5457e52e43b0c6c581d30cd WatchSource:0}: Error finding container 8076251e342f56c32051dade740deb9ad535be99e5457e52e43b0c6c581d30cd: Status 404 returned error can't find the container with id 8076251e342f56c32051dade740deb9ad535be99e5457e52e43b0c6c581d30cd
Jul 28 17:27:43.049741 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:43.049682    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-79sxg" event={"ID":"0487363e-1875-4e62-9289-141ccb5d0529","Type":"ContainerStarted","Data":"8076251e342f56c32051dade740deb9ad535be99e5457e52e43b0c6c581d30cd"}
Jul 28 17:27:43.051145 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:43.051115    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-xrfmj" event={"ID":"f45c022b-588c-474d-8d6a-ac6a6b340bda","Type":"ContainerStarted","Data":"8fbf6ebd965c1a056f72b23212ef876cb9c13e04ea5d7572cacb7147df02d7b6"}
Jul 28 17:27:47.073561 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.073520    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-xrfmj" event={"ID":"f45c022b-588c-474d-8d6a-ac6a6b340bda","Type":"ContainerStarted","Data":"eb1bf07940217ccfbe9754d90c77e63875599b2889f48cc89e9857c17ee92334"}
Jul 28 17:27:47.074038 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.073691    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:27:47.074915 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.074893    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7" event={"ID":"5424049e-4392-4be0-9016-cd08e91627d5","Type":"ContainerStarted","Data":"d0f07a42322463f8b913a3499e7d619cde78c0081e893e23d9167b29160c9abc"}
Jul 28 17:27:47.075014 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.075002    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:47.076212 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.076192    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-79sxg" event={"ID":"0487363e-1875-4e62-9289-141ccb5d0529","Type":"ContainerStarted","Data":"09cb81e799381449862a225f56af86b377a685a275e7c9a54863437230584306"}
Jul 28 17:27:47.076366 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.076311    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:27:47.102671 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.102620    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-xrfmj" podStartSLOduration=3.221068548 podStartE2EDuration="7.102605129s" totalImagesPullingTime="3.881536581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:47.101261186 +0000 UTC m=+289.047000349" watchObservedRunningTime="2026-07-28 17:27:47.102605129 +0000 UTC m=+289.048344309"
Jul 28 17:27:47.152748 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.152702    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7" podStartSLOduration=2.919797587 podStartE2EDuration="7.152686256s" totalImagesPullingTime="4.232888669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:47.151857183 +0000 UTC m=+289.097596358" watchObservedRunningTime="2026-07-28 17:27:47.152686256 +0000 UTC m=+289.098425417"
Jul 28 17:27:47.192105 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:47.192049    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-79sxg" podStartSLOduration=2.559041853 podStartE2EDuration="6.192032407s" totalImagesPullingTime="3.632990554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:47.191942348 +0000 UTC m=+289.137681511" watchObservedRunningTime="2026-07-28 17:27:47.192032407 +0000 UTC m=+289.137771568"
Jul 28 17:27:58.089848 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:58.089821    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-42dx7"
Jul 28 17:27:58.424402 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:58.424323    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:27:58.432142 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:58.432113    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:27:58.436465 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:27:58.436445    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 17:28:01.040973 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:28:01.040944    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-wchs9"
Jul 28 17:28:08.088495 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:28:08.088463    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-xrfmj"
Jul 28 17:28:08.088902 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:28:08.088540    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-79sxg"
Jul 28 17:29:06.516359 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.516321    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-2dddb"]
Jul 28 17:29:06.519126 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.519109    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.528289 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.528270    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 28 17:29:06.528419 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.528327    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 28 17:29:06.529181 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.529166    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 28 17:29:06.529256 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.529186    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-q95g8\""
Jul 28 17:29:06.591061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.591020    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-2dddb"]
Jul 28 17:29:06.613073 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.613047    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3df552-5968-4776-97f4-9247747dae39-cert\") pod \"kserve-controller-manager-cd4d44c67-2dddb\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") " pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.613206 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.613115    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f67k\" (UniqueName: \"kubernetes.io/projected/2a3df552-5968-4776-97f4-9247747dae39-kube-api-access-8f67k\") pod \"kserve-controller-manager-cd4d44c67-2dddb\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") " pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.620184 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.620153    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-tzlm2"]
Jul 28 17:29:06.622825 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.622809    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.626077 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.626001    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 28 17:29:06.626199 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.626091    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-r9pdw\""
Jul 28 17:29:06.656800 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.656772    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-tzlm2"]
Jul 28 17:29:06.713941 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.713905    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f67k\" (UniqueName: \"kubernetes.io/projected/2a3df552-5968-4776-97f4-9247747dae39-kube-api-access-8f67k\") pod \"kserve-controller-manager-cd4d44c67-2dddb\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") " pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.714112 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.713968    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n572\" (UniqueName: \"kubernetes.io/projected/5238996d-dbab-41a4-ae10-93b3209e3a09-kube-api-access-2n572\") pod \"seaweedfs-64568bcd49-tzlm2\" (UID: \"5238996d-dbab-41a4-ae10-93b3209e3a09\") " pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.714112 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.714003    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5238996d-dbab-41a4-ae10-93b3209e3a09-data\") pod \"seaweedfs-64568bcd49-tzlm2\" (UID: \"5238996d-dbab-41a4-ae10-93b3209e3a09\") " pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.714112 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.714055    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3df552-5968-4776-97f4-9247747dae39-cert\") pod \"kserve-controller-manager-cd4d44c67-2dddb\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") " pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.716371 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.716343    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3df552-5968-4776-97f4-9247747dae39-cert\") pod \"kserve-controller-manager-cd4d44c67-2dddb\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") " pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.775050 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.774980    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f67k\" (UniqueName: \"kubernetes.io/projected/2a3df552-5968-4776-97f4-9247747dae39-kube-api-access-8f67k\") pod \"kserve-controller-manager-cd4d44c67-2dddb\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") " pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.814566 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.814530    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n572\" (UniqueName: \"kubernetes.io/projected/5238996d-dbab-41a4-ae10-93b3209e3a09-kube-api-access-2n572\") pod \"seaweedfs-64568bcd49-tzlm2\" (UID: \"5238996d-dbab-41a4-ae10-93b3209e3a09\") " pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.814751 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.814578    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5238996d-dbab-41a4-ae10-93b3209e3a09-data\") pod \"seaweedfs-64568bcd49-tzlm2\" (UID: \"5238996d-dbab-41a4-ae10-93b3209e3a09\") " pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.814943 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.814926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5238996d-dbab-41a4-ae10-93b3209e3a09-data\") pod \"seaweedfs-64568bcd49-tzlm2\" (UID: \"5238996d-dbab-41a4-ae10-93b3209e3a09\") " pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.825763 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.825739    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n572\" (UniqueName: \"kubernetes.io/projected/5238996d-dbab-41a4-ae10-93b3209e3a09-kube-api-access-2n572\") pod \"seaweedfs-64568bcd49-tzlm2\" (UID: \"5238996d-dbab-41a4-ae10-93b3209e3a09\") " pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.829539 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.829523    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:06.932360 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.932334    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:06.987987 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.986649    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-2dddb"]
Jul 28 17:29:06.990894 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:29:06.990850    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3df552_5968_4776_97f4_9247747dae39.slice/crio-ed59cc0423aeba86374d0f0418988895b6023d5b18ccecb58519a83bf4530644 WatchSource:0}: Error finding container ed59cc0423aeba86374d0f0418988895b6023d5b18ccecb58519a83bf4530644: Status 404 returned error can't find the container with id ed59cc0423aeba86374d0f0418988895b6023d5b18ccecb58519a83bf4530644
Jul 28 17:29:06.992344 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:06.992294    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:29:07.078661 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:07.078533    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-tzlm2"]
Jul 28 17:29:07.081126 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:29:07.081096    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5238996d_dbab_41a4_ae10_93b3209e3a09.slice/crio-96667888e62193bdbabcff2b28f584a90d08ff40a32b5b13b588f9086dbaa069 WatchSource:0}: Error finding container 96667888e62193bdbabcff2b28f584a90d08ff40a32b5b13b588f9086dbaa069: Status 404 returned error can't find the container with id 96667888e62193bdbabcff2b28f584a90d08ff40a32b5b13b588f9086dbaa069
Jul 28 17:29:07.421466 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:07.421385    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb" event={"ID":"2a3df552-5968-4776-97f4-9247747dae39","Type":"ContainerStarted","Data":"ed59cc0423aeba86374d0f0418988895b6023d5b18ccecb58519a83bf4530644"}
Jul 28 17:29:07.422443 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:07.422421    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-tzlm2" event={"ID":"5238996d-dbab-41a4-ae10-93b3209e3a09","Type":"ContainerStarted","Data":"96667888e62193bdbabcff2b28f584a90d08ff40a32b5b13b588f9086dbaa069"}
Jul 28 17:29:11.443828 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:11.443796    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-tzlm2" event={"ID":"5238996d-dbab-41a4-ae10-93b3209e3a09","Type":"ContainerStarted","Data":"543b16996809d9663fb8b14cee21f4d4cd7b3a3d3f81888cbb94845277d10e1e"}
Jul 28 17:29:11.444318 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:11.443893    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:11.445261 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:11.445243    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb" event={"ID":"2a3df552-5968-4776-97f4-9247747dae39","Type":"ContainerStarted","Data":"51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d"}
Jul 28 17:29:11.445382 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:11.445370    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:11.496796 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:11.496740    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-tzlm2" podStartSLOduration=1.827686573 podStartE2EDuration="5.496723679s" totalImagesPullingTime="3.669037106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:29:11.495347673 +0000 UTC m=+373.441086855" watchObservedRunningTime="2026-07-28 17:29:11.496723679 +0000 UTC m=+373.442462841"
Jul 28 17:29:17.452211 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:17.452179    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-tzlm2"
Jul 28 17:29:17.542622 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:17.542573    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb" podStartSLOduration=7.875621213 podStartE2EDuration="11.542557096s" totalImagesPullingTime="3.666935883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:29:11.542727939 +0000 UTC m=+373.488467102" watchObservedRunningTime="2026-07-28 17:29:17.542557096 +0000 UTC m=+379.488296258"
Jul 28 17:29:42.455169 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:42.455137    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:52.747093 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.747059    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-2dddb"]
Jul 28 17:29:52.747484 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.747261    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb" podUID="2a3df552-5968-4776-97f4-9247747dae39" containerName="manager" containerID="cri-o://51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d" gracePeriod=10
Jul 28 17:29:52.790084 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.790058    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-47s2r"]
Jul 28 17:29:52.792253 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.792239    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:52.809704 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.809680    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-47s2r"]
Jul 28 17:29:52.916382 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.916355    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f87467e8-5552-4302-abf1-7c4440cff771-cert\") pod \"kserve-controller-manager-cd4d44c67-47s2r\" (UID: \"f87467e8-5552-4302-abf1-7c4440cff771\") " pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:52.916503 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.916462    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg6h9\" (UniqueName: \"kubernetes.io/projected/f87467e8-5552-4302-abf1-7c4440cff771-kube-api-access-bg6h9\") pod \"kserve-controller-manager-cd4d44c67-47s2r\" (UID: \"f87467e8-5552-4302-abf1-7c4440cff771\") " pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:52.986571 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:52.986546    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:53.017743 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.017668    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg6h9\" (UniqueName: \"kubernetes.io/projected/f87467e8-5552-4302-abf1-7c4440cff771-kube-api-access-bg6h9\") pod \"kserve-controller-manager-cd4d44c67-47s2r\" (UID: \"f87467e8-5552-4302-abf1-7c4440cff771\") " pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:53.017743 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.017732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f87467e8-5552-4302-abf1-7c4440cff771-cert\") pod \"kserve-controller-manager-cd4d44c67-47s2r\" (UID: \"f87467e8-5552-4302-abf1-7c4440cff771\") " pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:53.020042 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.020025    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f87467e8-5552-4302-abf1-7c4440cff771-cert\") pod \"kserve-controller-manager-cd4d44c67-47s2r\" (UID: \"f87467e8-5552-4302-abf1-7c4440cff771\") " pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:53.038795 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.038765    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg6h9\" (UniqueName: \"kubernetes.io/projected/f87467e8-5552-4302-abf1-7c4440cff771-kube-api-access-bg6h9\") pod \"kserve-controller-manager-cd4d44c67-47s2r\" (UID: \"f87467e8-5552-4302-abf1-7c4440cff771\") " pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:53.118115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.118082    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3df552-5968-4776-97f4-9247747dae39-cert\") pod \"2a3df552-5968-4776-97f4-9247747dae39\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") "
Jul 28 17:29:53.118284 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.118136    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f67k\" (UniqueName: \"kubernetes.io/projected/2a3df552-5968-4776-97f4-9247747dae39-kube-api-access-8f67k\") pod \"2a3df552-5968-4776-97f4-9247747dae39\" (UID: \"2a3df552-5968-4776-97f4-9247747dae39\") "
Jul 28 17:29:53.120151 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.120126    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a3df552-5968-4776-97f4-9247747dae39-cert" (OuterVolumeSpecName: "cert") pod "2a3df552-5968-4776-97f4-9247747dae39" (UID: "2a3df552-5968-4776-97f4-9247747dae39"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:29:53.120245 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.120223    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a3df552-5968-4776-97f4-9247747dae39-kube-api-access-8f67k" (OuterVolumeSpecName: "kube-api-access-8f67k") pod "2a3df552-5968-4776-97f4-9247747dae39" (UID: "2a3df552-5968-4776-97f4-9247747dae39"). InnerVolumeSpecName "kube-api-access-8f67k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:29:53.129844 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.129827    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:53.219573 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.219537    2556 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3df552-5968-4776-97f4-9247747dae39-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:29:53.219688 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.219605    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8f67k\" (UniqueName: \"kubernetes.io/projected/2a3df552-5968-4776-97f4-9247747dae39-kube-api-access-8f67k\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:29:53.267822 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.267774    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-47s2r"]
Jul 28 17:29:53.270074 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:29:53.270036    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87467e8_5552_4302_abf1_7c4440cff771.slice/crio-a9650e57fead687c8ddbed0553ec210d20fa971a4aae553c21bef4098956d4ca WatchSource:0}: Error finding container a9650e57fead687c8ddbed0553ec210d20fa971a4aae553c21bef4098956d4ca: Status 404 returned error can't find the container with id a9650e57fead687c8ddbed0553ec210d20fa971a4aae553c21bef4098956d4ca
Jul 28 17:29:53.630562 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.630538    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-cd4d44c67-47s2r" event={"ID":"f87467e8-5552-4302-abf1-7c4440cff771","Type":"ContainerStarted","Data":"a9650e57fead687c8ddbed0553ec210d20fa971a4aae553c21bef4098956d4ca"}
Jul 28 17:29:53.631631 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.631611    2556 generic.go:356] "Generic (PLEG): container finished" podID="2a3df552-5968-4776-97f4-9247747dae39" containerID="51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d" exitCode=0
Jul 28 17:29:53.631712 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.631655    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb" event={"ID":"2a3df552-5968-4776-97f4-9247747dae39","Type":"ContainerDied","Data":"51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d"}
Jul 28 17:29:53.631712 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.631690    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb" event={"ID":"2a3df552-5968-4776-97f4-9247747dae39","Type":"ContainerDied","Data":"ed59cc0423aeba86374d0f0418988895b6023d5b18ccecb58519a83bf4530644"}
Jul 28 17:29:53.631712 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.631694    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-cd4d44c67-2dddb"
Jul 28 17:29:53.631823 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.631711    2556 scope.go:117] "RemoveContainer" containerID="51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d"
Jul 28 17:29:53.640026 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.640005    2556 scope.go:117] "RemoveContainer" containerID="51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d"
Jul 28 17:29:53.640291 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:29:53.640269    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d\": container with ID starting with 51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d not found: ID does not exist" containerID="51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d"
Jul 28 17:29:53.640363 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.640297    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d"} err="failed to get container status \"51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d\": rpc error: code = NotFound desc = could not find container \"51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d\": container with ID starting with 51e64df1fabe230696072f7dd05d3ca90ad147669292fc98cb45b2045e8f375d not found: ID does not exist"
Jul 28 17:29:53.654543 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.654520    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-2dddb"]
Jul 28 17:29:53.660903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:53.660317    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-cd4d44c67-2dddb"]
Jul 28 17:29:54.589768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:54.589730    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a3df552-5968-4776-97f4-9247747dae39" path="/var/lib/kubelet/pods/2a3df552-5968-4776-97f4-9247747dae39/volumes"
Jul 28 17:29:54.637473 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:54.637440    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-cd4d44c67-47s2r" event={"ID":"f87467e8-5552-4302-abf1-7c4440cff771","Type":"ContainerStarted","Data":"d03b11038e21e5971539f19c6ce42ae0eb6cb26117433dd9f70229221d5244f4"}
Jul 28 17:29:54.637630 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:54.637583    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:29:54.656389 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:29:54.656348    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-cd4d44c67-47s2r" podStartSLOduration=2.338053909 podStartE2EDuration="2.656331615s" totalImagesPullingTime="318.277706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:29:54.655068297 +0000 UTC m=+416.600807503" watchObservedRunningTime="2026-07-28 17:29:54.656331615 +0000 UTC m=+416.602070793"
Jul 28 17:30:25.646690 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:25.646617    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-cd4d44c67-47s2r"
Jul 28 17:30:26.763085 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.763049    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-d5vl5"]
Jul 28 17:30:26.763651 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.763387    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3df552-5968-4776-97f4-9247747dae39" containerName="manager"
Jul 28 17:30:26.763651 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.763398    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3df552-5968-4776-97f4-9247747dae39" containerName="manager"
Jul 28 17:30:26.763651 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.763465    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3df552-5968-4776-97f4-9247747dae39" containerName="manager"
Jul 28 17:30:26.766480 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.766461    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:26.769465 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.769448    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 28 17:30:26.770297 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.770279    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-kh4n8\""
Jul 28 17:30:26.783459 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.783433    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-d5vl5"]
Jul 28 17:30:26.794623 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.794595    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-75d9cdbddf-922lz"]
Jul 28 17:30:26.797887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.797851    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:26.800497 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.800467    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 28 17:30:26.800595 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.800478    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-lckmc\""
Jul 28 17:30:26.807917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.807892    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-75d9cdbddf-922lz"]
Jul 28 17:30:26.908910 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.908862    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/917c768f-d5e2-4ceb-93d6-056987b1477f-tls-certs\") pod \"model-serving-api-7b954ffc95-d5vl5\" (UID: \"917c768f-d5e2-4ceb-93d6-056987b1477f\") " pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:26.909073 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.908962    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbrtr\" (UniqueName: \"kubernetes.io/projected/917c768f-d5e2-4ceb-93d6-056987b1477f-kube-api-access-wbrtr\") pod \"model-serving-api-7b954ffc95-d5vl5\" (UID: \"917c768f-d5e2-4ceb-93d6-056987b1477f\") " pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:26.909073 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.909004    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhvd\" (UniqueName: \"kubernetes.io/projected/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-kube-api-access-5qhvd\") pod \"odh-model-controller-75d9cdbddf-922lz\" (UID: \"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c\") " pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:26.909160 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:26.909071    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-cert\") pod \"odh-model-controller-75d9cdbddf-922lz\" (UID: \"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c\") " pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:27.009980 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.009940    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrtr\" (UniqueName: \"kubernetes.io/projected/917c768f-d5e2-4ceb-93d6-056987b1477f-kube-api-access-wbrtr\") pod \"model-serving-api-7b954ffc95-d5vl5\" (UID: \"917c768f-d5e2-4ceb-93d6-056987b1477f\") " pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:27.010155 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.010000    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhvd\" (UniqueName: \"kubernetes.io/projected/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-kube-api-access-5qhvd\") pod \"odh-model-controller-75d9cdbddf-922lz\" (UID: \"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c\") " pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:27.010155 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.010061    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-cert\") pod \"odh-model-controller-75d9cdbddf-922lz\" (UID: \"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c\") " pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:27.010155 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.010104    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/917c768f-d5e2-4ceb-93d6-056987b1477f-tls-certs\") pod \"model-serving-api-7b954ffc95-d5vl5\" (UID: \"917c768f-d5e2-4ceb-93d6-056987b1477f\") " pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:27.010274 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:30:27.010195    2556 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 17:30:27.010306 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:30:27.010281    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-cert podName:ebd3b847-c5e8-44a7-94eb-48a339ad2e3c nodeName:}" failed. No retries permitted until 2026-07-28 17:30:27.510260846 +0000 UTC m=+449.455999992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-cert") pod "odh-model-controller-75d9cdbddf-922lz" (UID: "ebd3b847-c5e8-44a7-94eb-48a339ad2e3c") : secret "odh-model-controller-webhook-cert" not found
Jul 28 17:30:27.012636 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.012611    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/917c768f-d5e2-4ceb-93d6-056987b1477f-tls-certs\") pod \"model-serving-api-7b954ffc95-d5vl5\" (UID: \"917c768f-d5e2-4ceb-93d6-056987b1477f\") " pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:27.019809 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.019748    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhvd\" (UniqueName: \"kubernetes.io/projected/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-kube-api-access-5qhvd\") pod \"odh-model-controller-75d9cdbddf-922lz\" (UID: \"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c\") " pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:27.020552 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.020529    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbrtr\" (UniqueName: \"kubernetes.io/projected/917c768f-d5e2-4ceb-93d6-056987b1477f-kube-api-access-wbrtr\") pod \"model-serving-api-7b954ffc95-d5vl5\" (UID: \"917c768f-d5e2-4ceb-93d6-056987b1477f\") " pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:27.077023 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.076991    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:27.210482 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.210458    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-d5vl5"]
Jul 28 17:30:27.212856 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:30:27.212832    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod917c768f_d5e2_4ceb_93d6_056987b1477f.slice/crio-30b0f65ddc589fd2699bbefb1e93f9a74c26229b580da193e12cca6b79f7eba0 WatchSource:0}: Error finding container 30b0f65ddc589fd2699bbefb1e93f9a74c26229b580da193e12cca6b79f7eba0: Status 404 returned error can't find the container with id 30b0f65ddc589fd2699bbefb1e93f9a74c26229b580da193e12cca6b79f7eba0
Jul 28 17:30:27.514450 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.514417    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-cert\") pod \"odh-model-controller-75d9cdbddf-922lz\" (UID: \"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c\") " pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:27.516670 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.516651    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebd3b847-c5e8-44a7-94eb-48a339ad2e3c-cert\") pod \"odh-model-controller-75d9cdbddf-922lz\" (UID: \"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c\") " pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:27.708331 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.708298    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:27.789388 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.789126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-d5vl5" event={"ID":"917c768f-d5e2-4ceb-93d6-056987b1477f","Type":"ContainerStarted","Data":"30b0f65ddc589fd2699bbefb1e93f9a74c26229b580da193e12cca6b79f7eba0"}
Jul 28 17:30:27.869649 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:30:27.869611    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd3b847_c5e8_44a7_94eb_48a339ad2e3c.slice/crio-ce2370882a3920bae9d512223106a04d5e54d588a85552e2e1f5afc0aa1a5fe0 WatchSource:0}: Error finding container ce2370882a3920bae9d512223106a04d5e54d588a85552e2e1f5afc0aa1a5fe0: Status 404 returned error can't find the container with id ce2370882a3920bae9d512223106a04d5e54d588a85552e2e1f5afc0aa1a5fe0
Jul 28 17:30:27.871223 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:27.871189    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-75d9cdbddf-922lz"]
Jul 28 17:30:28.797399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:28.797338    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-75d9cdbddf-922lz" event={"ID":"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c","Type":"ContainerStarted","Data":"ce2370882a3920bae9d512223106a04d5e54d588a85552e2e1f5afc0aa1a5fe0"}
Jul 28 17:30:31.812961 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.812928    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-868d45cc84-6cj4k"]
Jul 28 17:30:31.816694 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.816668    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:30:31.816839 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.816709    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-d5vl5" event={"ID":"917c768f-d5e2-4ceb-93d6-056987b1477f","Type":"ContainerStarted","Data":"e60c08631a8db5fa6bd0362f15ee05b5378929178ba4e8d750fff35936546f35"}
Jul 28 17:30:31.816839 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.816737    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:31.816839 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.816761    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-75d9cdbddf-922lz" event={"ID":"ebd3b847-c5e8-44a7-94eb-48a339ad2e3c","Type":"ContainerStarted","Data":"f494f21ed427d88b68f4095bb316549c9bbfa10fac9dac88e64c1711e98441d5"}
Jul 28 17:30:31.816839 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.816773    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:31.834115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.834081    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868d45cc84-6cj4k"]
Jul 28 17:30:31.851167 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.851119    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-d5vl5" podStartSLOduration=2.2684430349999998 podStartE2EDuration="5.851103799s" totalImagesPullingTime="3.582660764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:30:31.849600907 +0000 UTC m=+453.795340112" watchObservedRunningTime="2026-07-28 17:30:31.851103799 +0000 UTC m=+453.796842961"
Jul 28 17:30:31.880253 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.880204    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-75d9cdbddf-922lz" podStartSLOduration=2.90726154 podStartE2EDuration="5.880187939s" totalImagesPullingTime="2.972926399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:30:31.879103599 +0000 UTC m=+453.824842777" watchObservedRunningTime="2026-07-28 17:30:31.880187939 +0000 UTC m=+453.825927124"
Jul 28 17:30:31.957308 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.957276    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-service-ca\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:31.957469 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.957450    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-trusted-ca-bundle\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:31.957519 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.957507    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm65r\" (UniqueName: \"kubernetes.io/projected/21a0c0ba-5fb3-4524-87fa-e802174c62a5-kube-api-access-hm65r\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:31.957554 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.957537    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-serving-cert\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:31.957601 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.957559    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-oauth-serving-cert\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:31.957638 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.957614    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-oauth-config\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:31.957680 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:31.957662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-config\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.058528 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.058492    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-config\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.058719 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.058541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-service-ca\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.058719 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.058578    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-trusted-ca-bundle\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.058796 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.058738    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm65r\" (UniqueName: \"kubernetes.io/projected/21a0c0ba-5fb3-4524-87fa-e802174c62a5-kube-api-access-hm65r\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.058796 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.058788    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-serving-cert\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.058917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.058812    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-oauth-serving-cert\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.058917 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.058848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-oauth-config\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.059374 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.059339    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-config\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.059493 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.059375    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-service-ca\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.059493 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.059478    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-trusted-ca-bundle\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.059608 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.059495    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21a0c0ba-5fb3-4524-87fa-e802174c62a5-oauth-serving-cert\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.061226 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.061195    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-serving-cert\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.061328 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.061198    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21a0c0ba-5fb3-4524-87fa-e802174c62a5-console-oauth-config\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.072587 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.072534    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm65r\" (UniqueName: \"kubernetes.io/projected/21a0c0ba-5fb3-4524-87fa-e802174c62a5-kube-api-access-hm65r\") pod \"console-868d45cc84-6cj4k\" (UID: \"21a0c0ba-5fb3-4524-87fa-e802174c62a5\") " pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.127371 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.127335    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:32.267977 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.267953    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868d45cc84-6cj4k"]
Jul 28 17:30:32.270425 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:30:32.270398    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a0c0ba_5fb3_4524_87fa_e802174c62a5.slice/crio-bc22da19f85e27e06c5de649a145ed9b83ae50ea599e16baa4693a91c482a6a2 WatchSource:0}: Error finding container bc22da19f85e27e06c5de649a145ed9b83ae50ea599e16baa4693a91c482a6a2: Status 404 returned error can't find the container with id bc22da19f85e27e06c5de649a145ed9b83ae50ea599e16baa4693a91c482a6a2
Jul 28 17:30:32.822674 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.822634    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868d45cc84-6cj4k" event={"ID":"21a0c0ba-5fb3-4524-87fa-e802174c62a5","Type":"ContainerStarted","Data":"ad80d46f4d6bf466da8daee8a89735c44a5ed5056b2bda03311ac479a644c47e"}
Jul 28 17:30:32.823159 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.822694    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868d45cc84-6cj4k" event={"ID":"21a0c0ba-5fb3-4524-87fa-e802174c62a5","Type":"ContainerStarted","Data":"bc22da19f85e27e06c5de649a145ed9b83ae50ea599e16baa4693a91c482a6a2"}
Jul 28 17:30:32.844254 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:32.844201    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-868d45cc84-6cj4k" podStartSLOduration=1.8441839660000001 podStartE2EDuration="1.844183966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:30:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:30:32.841720123 +0000 UTC m=+454.787459304" watchObservedRunningTime="2026-07-28 17:30:32.844183966 +0000 UTC m=+454.789923128"
Jul 28 17:30:42.128360 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:42.128321    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:42.128750 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:42.128408    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:42.133009 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:42.132986    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:42.825503 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:42.825476    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-75d9cdbddf-922lz"
Jul 28 17:30:42.871624 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:42.871598    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-868d45cc84-6cj4k"
Jul 28 17:30:42.932852 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:42.932816    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64569bcf6b-7k2qc"]
Jul 28 17:30:43.832130 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:30:43.832105    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-d5vl5"
Jul 28 17:31:07.953088 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:07.953046    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64569bcf6b-7k2qc" podUID="77b07156-b840-43a1-8e5c-3febff7f6bd8" containerName="console" containerID="cri-o://1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614" gracePeriod=15
Jul 28 17:31:08.198439 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.198415    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:31:08.279041 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.278949    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-trusted-ca-bundle\") pod \"77b07156-b840-43a1-8e5c-3febff7f6bd8\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") "
Jul 28 17:31:08.279041 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.278997    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-config\") pod \"77b07156-b840-43a1-8e5c-3febff7f6bd8\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") "
Jul 28 17:31:08.279041 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279026    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-serving-cert\") pod \"77b07156-b840-43a1-8e5c-3febff7f6bd8\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") "
Jul 28 17:31:08.279278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279061    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-oauth-serving-cert\") pod \"77b07156-b840-43a1-8e5c-3febff7f6bd8\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") "
Jul 28 17:31:08.279278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279096    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7tqr\" (UniqueName: \"kubernetes.io/projected/77b07156-b840-43a1-8e5c-3febff7f6bd8-kube-api-access-d7tqr\") pod \"77b07156-b840-43a1-8e5c-3febff7f6bd8\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") "
Jul 28 17:31:08.279380 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279298    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-service-ca\") pod \"77b07156-b840-43a1-8e5c-3febff7f6bd8\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") "
Jul 28 17:31:08.279430 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279375    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "77b07156-b840-43a1-8e5c-3febff7f6bd8" (UID: "77b07156-b840-43a1-8e5c-3febff7f6bd8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:31:08.279430 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279390    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-oauth-config\") pod \"77b07156-b840-43a1-8e5c-3febff7f6bd8\" (UID: \"77b07156-b840-43a1-8e5c-3febff7f6bd8\") "
Jul 28 17:31:08.279525 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279464    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "77b07156-b840-43a1-8e5c-3febff7f6bd8" (UID: "77b07156-b840-43a1-8e5c-3febff7f6bd8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:31:08.279525 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279487    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-config" (OuterVolumeSpecName: "console-config") pod "77b07156-b840-43a1-8e5c-3febff7f6bd8" (UID: "77b07156-b840-43a1-8e5c-3febff7f6bd8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:31:08.279734 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279716    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-oauth-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:31:08.279794 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279728    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-service-ca" (OuterVolumeSpecName: "service-ca") pod "77b07156-b840-43a1-8e5c-3febff7f6bd8" (UID: "77b07156-b840-43a1-8e5c-3febff7f6bd8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:31:08.279794 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279739    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-trusted-ca-bundle\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:31:08.279794 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.279753    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:31:08.281330 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.281310    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "77b07156-b840-43a1-8e5c-3febff7f6bd8" (UID: "77b07156-b840-43a1-8e5c-3febff7f6bd8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:31:08.281708 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.281677    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "77b07156-b840-43a1-8e5c-3febff7f6bd8" (UID: "77b07156-b840-43a1-8e5c-3febff7f6bd8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:31:08.281708 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.281690    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b07156-b840-43a1-8e5c-3febff7f6bd8-kube-api-access-d7tqr" (OuterVolumeSpecName: "kube-api-access-d7tqr") pod "77b07156-b840-43a1-8e5c-3febff7f6bd8" (UID: "77b07156-b840-43a1-8e5c-3febff7f6bd8"). InnerVolumeSpecName "kube-api-access-d7tqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:31:08.380369 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.380334    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7tqr\" (UniqueName: \"kubernetes.io/projected/77b07156-b840-43a1-8e5c-3febff7f6bd8-kube-api-access-d7tqr\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:31:08.380369 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.380364    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77b07156-b840-43a1-8e5c-3febff7f6bd8-service-ca\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:31:08.380369 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.380377    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-oauth-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:31:08.380605 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.380389    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77b07156-b840-43a1-8e5c-3febff7f6bd8-console-serving-cert\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:31:08.989582 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.989553    2556 generic.go:356] "Generic (PLEG): container finished" podID="77b07156-b840-43a1-8e5c-3febff7f6bd8" containerID="1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614" exitCode=0
Jul 28 17:31:08.990051 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.989595    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64569bcf6b-7k2qc" event={"ID":"77b07156-b840-43a1-8e5c-3febff7f6bd8","Type":"ContainerDied","Data":"1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614"}
Jul 28 17:31:08.990051 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.989606    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64569bcf6b-7k2qc"
Jul 28 17:31:08.990051 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.989634    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64569bcf6b-7k2qc" event={"ID":"77b07156-b840-43a1-8e5c-3febff7f6bd8","Type":"ContainerDied","Data":"1ada6f6c1bc9fe7b20ee665c435f5382e14b7709ebffce0dc66cdf38f31377d8"}
Jul 28 17:31:08.990051 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.989655    2556 scope.go:117] "RemoveContainer" containerID="1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614"
Jul 28 17:31:08.998048 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.998032    2556 scope.go:117] "RemoveContainer" containerID="1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614"
Jul 28 17:31:08.998292 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:31:08.998271    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614\": container with ID starting with 1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614 not found: ID does not exist" containerID="1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614"
Jul 28 17:31:08.998339 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:08.998301    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614"} err="failed to get container status \"1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614\": rpc error: code = NotFound desc = could not find container \"1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614\": container with ID starting with 1bc7266fdcdc16725ddb2d077745f64912c77f6fd4243f0512282b4527b04614 not found: ID does not exist"
Jul 28 17:31:09.006508 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:09.006485    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64569bcf6b-7k2qc"]
Jul 28 17:31:09.013114 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:09.013094    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64569bcf6b-7k2qc"]
Jul 28 17:31:10.590652 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:31:10.590618    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b07156-b840-43a1-8e5c-3febff7f6bd8" path="/var/lib/kubelet/pods/77b07156-b840-43a1-8e5c-3febff7f6bd8/volumes"
Jul 28 17:32:39.530887 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.530849    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"]
Jul 28 17:32:39.531468 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.531248    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b07156-b840-43a1-8e5c-3febff7f6bd8" containerName="console"
Jul 28 17:32:39.531468 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.531263    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b07156-b840-43a1-8e5c-3febff7f6bd8" containerName="console"
Jul 28 17:32:39.531468 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.531333    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b07156-b840-43a1-8e5c-3febff7f6bd8" containerName="console"
Jul 28 17:32:39.534640 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.534621    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.537095 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.537074    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 28 17:32:39.537219 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.537127    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0903-predictor-serving-cert\""
Jul 28 17:32:39.537295 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.537077    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7f5hv\""
Jul 28 17:32:39.537530 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.537508    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 28 17:32:39.537627 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.537605    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0903-kube-rbac-proxy-sar-config\""
Jul 28 17:32:39.543760 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.543741    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"]
Jul 28 17:32:39.580660 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.580616    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.580801 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.580675    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.580801 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.580718    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hsh7\" (UniqueName: \"kubernetes.io/projected/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-api-access-8hsh7\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.681615 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.681578    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hsh7\" (UniqueName: \"kubernetes.io/projected/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-api-access-8hsh7\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.681809 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.681695    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.681809 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.681748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.681961 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:32:39.681828    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-f0903-predictor-serving-cert: secret "error-404-isvc-f0903-predictor-serving-cert" not found
Jul 28 17:32:39.681961 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:32:39.681927    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls podName:f08dd03d-55a4-43ec-8a2a-6952e8d6a038 nodeName:}" failed. No retries permitted until 2026-07-28 17:32:40.181905878 +0000 UTC m=+582.127645034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls") pod "error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" (UID: "f08dd03d-55a4-43ec-8a2a-6952e8d6a038") : secret "error-404-isvc-f0903-predictor-serving-cert" not found
Jul 28 17:32:39.682541 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.682516    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:39.693517 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:39.693480    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hsh7\" (UniqueName: \"kubernetes.io/projected/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-api-access-8hsh7\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:40.186361 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:40.186318    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:40.188810 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:40.188784    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls\") pod \"error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") " pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:40.446687 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:40.446599    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:40.604051 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:40.604027    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"]
Jul 28 17:32:41.436971 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:41.436860    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" event={"ID":"f08dd03d-55a4-43ec-8a2a-6952e8d6a038","Type":"ContainerStarted","Data":"38f18083a224c043fa15f3bd5d7b55ba2b963b1919c72f76af722e93e28c239e"}
Jul 28 17:32:52.506766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:52.506639    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" event={"ID":"f08dd03d-55a4-43ec-8a2a-6952e8d6a038","Type":"ContainerStarted","Data":"7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181"}
Jul 28 17:32:54.520471 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:54.520430    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" event={"ID":"f08dd03d-55a4-43ec-8a2a-6952e8d6a038","Type":"ContainerStarted","Data":"adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0"}
Jul 28 17:32:54.520926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:54.520594    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:54.539581 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:54.539532    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podStartSLOduration=2.129505649 podStartE2EDuration="15.53951717s" totalImagesPullingTime="13.410011521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:32:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:32:54.537767948 +0000 UTC m=+596.483507110" watchObservedRunningTime="2026-07-28 17:32:54.53951717 +0000 UTC m=+596.485256332"
Jul 28 17:32:55.525322 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:55.525280    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:32:55.526979 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:55.526953    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 28 17:32:56.531020 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:56.530983    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 28 17:32:58.460825 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:58.460796    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:32:58.462242 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:32:58.462218    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:33:01.536053 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:33:01.536025    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:33:01.538336 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:33:01.536446    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 28 17:33:11.536822 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:33:11.536781    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 28 17:33:21.536829 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:33:21.536745    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 28 17:33:31.537148 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:33:31.537106    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 28 17:33:41.537791 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:33:41.537760    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:34:13.590701 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.590662    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"]
Jul 28 17:34:13.593330 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.593312    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.595589 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.595565    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-22e0e-predictor-serving-cert\""
Jul 28 17:34:13.595713 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.595587    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-22e0e-kube-rbac-proxy-sar-config\""
Jul 28 17:34:13.603497 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.603468    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"]
Jul 28 17:34:13.632352 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.632321    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"]
Jul 28 17:34:13.632585 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.632561    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container" containerID="cri-o://7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181" gracePeriod=30
Jul 28 17:34:13.632667 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.632594    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kube-rbac-proxy" containerID="cri-o://adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0" gracePeriod=30
Jul 28 17:34:13.737821 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.737791    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxzkf\" (UniqueName: \"kubernetes.io/projected/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-api-access-bxzkf\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.738041 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.737839    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.738041 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.737968    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.839037 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.839000    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.839199 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.839151    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxzkf\" (UniqueName: \"kubernetes.io/projected/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-api-access-bxzkf\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.839199 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:34:13.839157    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-serving-cert: secret "error-404-isvc-22e0e-predictor-serving-cert" not found
Jul 28 17:34:13.839199 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.839191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.839329 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:34:13.839217    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls podName:040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:14.339197542 +0000 UTC m=+676.284936684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls") pod "error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" (UID: "040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9") : secret "error-404-isvc-22e0e-predictor-serving-cert" not found
Jul 28 17:34:13.839746 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.839729    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.849197 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.849137    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxzkf\" (UniqueName: \"kubernetes.io/projected/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-api-access-bxzkf\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:13.892464 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.892432    2556 generic.go:356] "Generic (PLEG): container finished" podID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerID="adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0" exitCode=2
Jul 28 17:34:13.892625 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:13.892508    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" event={"ID":"f08dd03d-55a4-43ec-8a2a-6952e8d6a038","Type":"ContainerDied","Data":"adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0"}
Jul 28 17:34:14.344888 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.344830    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:14.347332 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.347305    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls\") pod \"error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") " pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:14.505835 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.505787    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:14.637809 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.637766    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"]
Jul 28 17:34:14.640421 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:34:14.640392    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040fce8d_7c1d_4b53_b4af_e6ddc6ebd1e9.slice/crio-ab635a665419497684200ff794bb44f27b0aeea33737e55acecd940768986a16 WatchSource:0}: Error finding container ab635a665419497684200ff794bb44f27b0aeea33737e55acecd940768986a16: Status 404 returned error can't find the container with id ab635a665419497684200ff794bb44f27b0aeea33737e55acecd940768986a16
Jul 28 17:34:14.899908 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.899798    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" event={"ID":"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9","Type":"ContainerStarted","Data":"17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8"}
Jul 28 17:34:14.899908 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.899857    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" event={"ID":"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9","Type":"ContainerStarted","Data":"4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47"}
Jul 28 17:34:14.900140 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.899907    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" event={"ID":"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9","Type":"ContainerStarted","Data":"ab635a665419497684200ff794bb44f27b0aeea33737e55acecd940768986a16"}
Jul 28 17:34:14.900140 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.900044    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:14.900140 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.900080    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:14.901416 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.901388    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 28 17:34:14.917105 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:14.917057    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podStartSLOduration=1.917042047 podStartE2EDuration="1.917042047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:14.915789749 +0000 UTC m=+676.861528933" watchObservedRunningTime="2026-07-28 17:34:14.917042047 +0000 UTC m=+676.862781210"
Jul 28 17:34:15.904798 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:15.904765    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 28 17:34:16.531492 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.531442    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Jul 28 17:34:16.784674 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.784614    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:34:16.866993 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.866959    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls\") pod \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") "
Jul 28 17:34:16.866993 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.866995    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-rbac-proxy-sar-config\") pod \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") "
Jul 28 17:34:16.867215 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.867039    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hsh7\" (UniqueName: \"kubernetes.io/projected/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-api-access-8hsh7\") pod \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\" (UID: \"f08dd03d-55a4-43ec-8a2a-6952e8d6a038\") "
Jul 28 17:34:16.867382 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.867356    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f08dd03d-55a4-43ec-8a2a-6952e8d6a038" (UID: "f08dd03d-55a4-43ec-8a2a-6952e8d6a038"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:34:16.869108 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.869089    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-api-access-8hsh7" (OuterVolumeSpecName: "kube-api-access-8hsh7") pod "f08dd03d-55a4-43ec-8a2a-6952e8d6a038" (UID: "f08dd03d-55a4-43ec-8a2a-6952e8d6a038"). InnerVolumeSpecName "kube-api-access-8hsh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:34:16.869245 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.869227    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f08dd03d-55a4-43ec-8a2a-6952e8d6a038" (UID: "f08dd03d-55a4-43ec-8a2a-6952e8d6a038"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:34:16.910788 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.910756    2556 generic.go:356] "Generic (PLEG): container finished" podID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerID="7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181" exitCode=0
Jul 28 17:34:16.911259 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.910814    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"
Jul 28 17:34:16.911259 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.910833    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" event={"ID":"f08dd03d-55a4-43ec-8a2a-6952e8d6a038","Type":"ContainerDied","Data":"7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181"}
Jul 28 17:34:16.911259 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.910910    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd" event={"ID":"f08dd03d-55a4-43ec-8a2a-6952e8d6a038","Type":"ContainerDied","Data":"38f18083a224c043fa15f3bd5d7b55ba2b963b1919c72f76af722e93e28c239e"}
Jul 28 17:34:16.911259 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.910937    2556 scope.go:117] "RemoveContainer" containerID="adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0"
Jul 28 17:34:16.919471 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.919452    2556 scope.go:117] "RemoveContainer" containerID="7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181"
Jul 28 17:34:16.926810 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.926794    2556 scope.go:117] "RemoveContainer" containerID="adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0"
Jul 28 17:34:16.927054 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:34:16.927037    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0\": container with ID starting with adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0 not found: ID does not exist" containerID="adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0"
Jul 28 17:34:16.927105 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.927064    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0"} err="failed to get container status \"adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0\": rpc error: code = NotFound desc = could not find container \"adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0\": container with ID starting with adbedf5d73b547541b1cb8f65b4197475cddf8887a4e6a5d1eeb518c69734cd0 not found: ID does not exist"
Jul 28 17:34:16.927105 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.927080    2556 scope.go:117] "RemoveContainer" containerID="7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181"
Jul 28 17:34:16.927328 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:34:16.927309    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181\": container with ID starting with 7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181 not found: ID does not exist" containerID="7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181"
Jul 28 17:34:16.927382 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.927339    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181"} err="failed to get container status \"7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181\": rpc error: code = NotFound desc = could not find container \"7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181\": container with ID starting with 7836da9d8f9bfe5642799f7e8484e81b096cd493e22ad16947d5d9dc3cf90181 not found: ID does not exist"
Jul 28 17:34:16.931461 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.931434    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"]
Jul 28 17:34:16.935135 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.935114    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0903-predictor-5f4f5b79bb-nlqzd"]
Jul 28 17:34:16.968181 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.968160    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:34:16.968181 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.968180    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:34:16.968318 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:16.968189    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hsh7\" (UniqueName: \"kubernetes.io/projected/f08dd03d-55a4-43ec-8a2a-6952e8d6a038-kube-api-access-8hsh7\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:34:18.590929 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:18.590889    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" path="/var/lib/kubelet/pods/f08dd03d-55a4-43ec-8a2a-6952e8d6a038/volumes"
Jul 28 17:34:20.909224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:20.909198    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:34:20.909683 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:20.909652    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 28 17:34:30.910126 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:30.910088    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 28 17:34:40.909695 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:40.909659    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 28 17:34:50.910531 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:34:50.910446    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 28 17:35:00.910697 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:00.910666    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:35:23.438167 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.438126    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"]
Jul 28 17:35:23.438738 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.438639    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kube-rbac-proxy"
Jul 28 17:35:23.438738 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.438653    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kube-rbac-proxy"
Jul 28 17:35:23.438738 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.438676    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container"
Jul 28 17:35:23.438738 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.438686    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container"
Jul 28 17:35:23.438955 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.438780    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kserve-container"
Jul 28 17:35:23.438955 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.438794    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08dd03d-55a4-43ec-8a2a-6952e8d6a038" containerName="kube-rbac-proxy"
Jul 28 17:35:23.441336 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.441318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.443392 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.443366    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b1dfe-predictor-serving-cert\""
Jul 28 17:35:23.443515 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.443417    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b1dfe-kube-rbac-proxy-sar-config\""
Jul 28 17:35:23.453160 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.453136    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"]
Jul 28 17:35:23.520694 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.520659    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.520865 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.520730    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7smm\" (UniqueName: \"kubernetes.io/projected/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-api-access-j7smm\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.520865 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.520797    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.621885 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.621843    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7smm\" (UniqueName: \"kubernetes.io/projected/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-api-access-j7smm\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.621885 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.621892    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.622118 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.621959    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.622118 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:35:23.621996    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-serving-cert: secret "error-404-isvc-b1dfe-predictor-serving-cert" not found
Jul 28 17:35:23.622118 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:35:23.622067    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls podName:74662a03-b254-4f2c-ac7c-d5d7328879e3 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:24.122047867 +0000 UTC m=+746.067787012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls") pod "error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" (UID: "74662a03-b254-4f2c-ac7c-d5d7328879e3") : secret "error-404-isvc-b1dfe-predictor-serving-cert" not found
Jul 28 17:35:23.622601 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.622582    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:23.632651 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:23.632628    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7smm\" (UniqueName: \"kubernetes.io/projected/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-api-access-j7smm\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:24.127068 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:24.127035    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:24.127285 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:35:24.127160    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-serving-cert: secret "error-404-isvc-b1dfe-predictor-serving-cert" not found
Jul 28 17:35:24.127285 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:35:24.127222    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls podName:74662a03-b254-4f2c-ac7c-d5d7328879e3 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:25.127205384 +0000 UTC m=+747.072944526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls") pod "error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" (UID: "74662a03-b254-4f2c-ac7c-d5d7328879e3") : secret "error-404-isvc-b1dfe-predictor-serving-cert" not found
Jul 28 17:35:25.136553 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:25.136511    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:25.138888 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:25.138854    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls\") pod \"error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") " pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:25.252819 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:25.252787    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:25.381891 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:25.381795    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"]
Jul 28 17:35:25.384228 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:35:25.384199    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74662a03_b254_4f2c_ac7c_d5d7328879e3.slice/crio-292e12f2449694c28c42bc19253c77ffe2b8dc3305249227595dcb231db2c648 WatchSource:0}: Error finding container 292e12f2449694c28c42bc19253c77ffe2b8dc3305249227595dcb231db2c648: Status 404 returned error can't find the container with id 292e12f2449694c28c42bc19253c77ffe2b8dc3305249227595dcb231db2c648
Jul 28 17:35:26.251366 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:26.251330    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" event={"ID":"74662a03-b254-4f2c-ac7c-d5d7328879e3","Type":"ContainerStarted","Data":"8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a"}
Jul 28 17:35:26.251766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:26.251388    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" event={"ID":"74662a03-b254-4f2c-ac7c-d5d7328879e3","Type":"ContainerStarted","Data":"1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec"}
Jul 28 17:35:26.251766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:26.251411    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" event={"ID":"74662a03-b254-4f2c-ac7c-d5d7328879e3","Type":"ContainerStarted","Data":"292e12f2449694c28c42bc19253c77ffe2b8dc3305249227595dcb231db2c648"}
Jul 28 17:35:26.251766 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:26.251435    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:26.271454 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:26.271408    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podStartSLOduration=3.271390561 podStartE2EDuration="3.271390561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:26.270804348 +0000 UTC m=+748.216543544" watchObservedRunningTime="2026-07-28 17:35:26.271390561 +0000 UTC m=+748.217129724"
Jul 28 17:35:27.255637 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:27.255602    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:27.257038 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:27.257011    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 28 17:35:28.260280 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:28.260241    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 28 17:35:33.264316 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:33.264285    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:35:33.264733 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:33.264709    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 28 17:35:43.265589 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:43.265551    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 28 17:35:53.265067 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:35:53.265023    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 28 17:36:03.265540 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:36:03.265480    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 28 17:36:13.265946 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:36:13.265917    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:37:58.491636 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:37:58.491597    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:37:58.492141 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:37:58.491643    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:42:58.520746 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:42:58.520718    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:42:58.522504 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:42:58.522480    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:43:28.406759 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.406726    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"]
Jul 28 17:43:28.410411 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.410393    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.414432 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.414327    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8b8b5-predictor-serving-cert\""
Jul 28 17:43:28.414432 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.414358    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8b8b5-kube-rbac-proxy-sar-config\""
Jul 28 17:43:28.427310 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.427288    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"]
Jul 28 17:43:28.506962 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.506933    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"]
Jul 28 17:43:28.507198 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.507155    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" containerID="cri-o://4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47" gracePeriod=30
Jul 28 17:43:28.507260 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.507188    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kube-rbac-proxy" containerID="cri-o://17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8" gracePeriod=30
Jul 28 17:43:28.531143 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.531118    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abeb40e-8792-44da-a994-2ae33123547e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.531286 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.531252    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.531354 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.531323    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t54l9\" (UniqueName: \"kubernetes.io/projected/0abeb40e-8792-44da-a994-2ae33123547e-kube-api-access-t54l9\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.632497 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.632466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.632626 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.632510    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t54l9\" (UniqueName: \"kubernetes.io/projected/0abeb40e-8792-44da-a994-2ae33123547e-kube-api-access-t54l9\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.632626 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.632567    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abeb40e-8792-44da-a994-2ae33123547e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.632742 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:43:28.632625    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-serving-cert: secret "error-404-isvc-8b8b5-predictor-serving-cert" not found
Jul 28 17:43:28.632742 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:43:28.632689    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls podName:0abeb40e-8792-44da-a994-2ae33123547e nodeName:}" failed. No retries permitted until 2026-07-28 17:43:29.132668513 +0000 UTC m=+1231.078407669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls") pod "error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" (UID: "0abeb40e-8792-44da-a994-2ae33123547e") : secret "error-404-isvc-8b8b5-predictor-serving-cert" not found
Jul 28 17:43:28.633202 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.633179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abeb40e-8792-44da-a994-2ae33123547e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:28.642384 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:28.642360    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t54l9\" (UniqueName: \"kubernetes.io/projected/0abeb40e-8792-44da-a994-2ae33123547e-kube-api-access-t54l9\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:29.137493 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.137447    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:29.139945 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.139921    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls\") pod \"error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") " pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:29.321199 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.321167    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:29.461430 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.461404    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"]
Jul 28 17:43:29.463860 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:43:29.463834    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abeb40e_8792_44da_a994_2ae33123547e.slice/crio-b489f2664601c18875e72eebea9db96ff1f7fc3e210a88719da54a7a6dbe7d82 WatchSource:0}: Error finding container b489f2664601c18875e72eebea9db96ff1f7fc3e210a88719da54a7a6dbe7d82: Status 404 returned error can't find the container with id b489f2664601c18875e72eebea9db96ff1f7fc3e210a88719da54a7a6dbe7d82
Jul 28 17:43:29.557446 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.557419    2556 generic.go:356] "Generic (PLEG): container finished" podID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerID="17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8" exitCode=2
Jul 28 17:43:29.557589 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.557485    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" event={"ID":"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9","Type":"ContainerDied","Data":"17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8"}
Jul 28 17:43:29.558916 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.558892    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" event={"ID":"0abeb40e-8792-44da-a994-2ae33123547e","Type":"ContainerStarted","Data":"53e2c5dcfaa77c134ce78e349c41c4f86a12649e3886bfe332712e928431204e"}
Jul 28 17:43:29.559018 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:29.558942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" event={"ID":"0abeb40e-8792-44da-a994-2ae33123547e","Type":"ContainerStarted","Data":"b489f2664601c18875e72eebea9db96ff1f7fc3e210a88719da54a7a6dbe7d82"}
Jul 28 17:43:30.565633 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:30.565595    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" event={"ID":"0abeb40e-8792-44da-a994-2ae33123547e","Type":"ContainerStarted","Data":"f7c123c5b8d9b64b9e4b4c7bc72bf9c76cb094e9334841dfe65f4db51bfbaf7c"}
Jul 28 17:43:30.566027 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:30.565694    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:30.589610 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:30.589552    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podStartSLOduration=2.58953191 podStartE2EDuration="2.58953191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:43:30.588455091 +0000 UTC m=+1232.534194253" watchObservedRunningTime="2026-07-28 17:43:30.58953191 +0000 UTC m=+1232.535271077"
Jul 28 17:43:30.905271 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:30.905183    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.39:8643/healthz\": dial tcp 10.134.0.39:8643: connect: connection refused"
Jul 28 17:43:30.910583 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:30.910555    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 28 17:43:31.570657 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:31.570621    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:31.572012 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:31.571985    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 28 17:43:31.869731 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:31.869707    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:43:32.063588 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.063555    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-rbac-proxy-sar-config\") pod \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") "
Jul 28 17:43:32.063792 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.063625    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls\") pod \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") "
Jul 28 17:43:32.063792 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.063658    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxzkf\" (UniqueName: \"kubernetes.io/projected/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-api-access-bxzkf\") pod \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\" (UID: \"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9\") "
Jul 28 17:43:32.064027 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.064002    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" (UID: "040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:43:32.065693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.065655    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" (UID: "040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:43:32.065693 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.065681    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-api-access-bxzkf" (OuterVolumeSpecName: "kube-api-access-bxzkf") pod "040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" (UID: "040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9"). InnerVolumeSpecName "kube-api-access-bxzkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:43:32.164946 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.164826    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:43:32.164946 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.164895    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:43:32.164946 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.164912    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxzkf\" (UniqueName: \"kubernetes.io/projected/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9-kube-api-access-bxzkf\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:43:32.576476 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.576443    2556 generic.go:356] "Generic (PLEG): container finished" podID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerID="4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47" exitCode=0
Jul 28 17:43:32.576931 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.576492    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"
Jul 28 17:43:32.576931 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.576529    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" event={"ID":"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9","Type":"ContainerDied","Data":"4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47"}
Jul 28 17:43:32.576931 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.576594    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k" event={"ID":"040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9","Type":"ContainerDied","Data":"ab635a665419497684200ff794bb44f27b0aeea33737e55acecd940768986a16"}
Jul 28 17:43:32.576931 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.576632    2556 scope.go:117] "RemoveContainer" containerID="17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8"
Jul 28 17:43:32.577166 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.577037    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 28 17:43:32.585613 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.585591    2556 scope.go:117] "RemoveContainer" containerID="4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47"
Jul 28 17:43:32.593973 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.593952    2556 scope.go:117] "RemoveContainer" containerID="17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8"
Jul 28 17:43:32.594220 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:43:32.594201    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8\": container with ID starting with 17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8 not found: ID does not exist" containerID="17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8"
Jul 28 17:43:32.594331 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.594229    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8"} err="failed to get container status \"17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8\": rpc error: code = NotFound desc = could not find container \"17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8\": container with ID starting with 17c58f644a51c388d9c99fd2e16eefae2100acb676f9c78c4acac7e8b3da7ed8 not found: ID does not exist"
Jul 28 17:43:32.594331 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.594245    2556 scope.go:117] "RemoveContainer" containerID="4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47"
Jul 28 17:43:32.594501 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:43:32.594486    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47\": container with ID starting with 4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47 not found: ID does not exist" containerID="4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47"
Jul 28 17:43:32.594542 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.594504    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47"} err="failed to get container status \"4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47\": rpc error: code = NotFound desc = could not find container \"4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47\": container with ID starting with 4b0944d2ce5a4c88f468929f8be7316a6d5c2772d929b204938daff14e444f47 not found: ID does not exist"
Jul 28 17:43:32.601299 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.601278    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"]
Jul 28 17:43:32.612608 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:32.612587    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-22e0e-predictor-6bc85ccf58-kfk2k"]
Jul 28 17:43:34.591063 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:34.591030    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" path="/var/lib/kubelet/pods/040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9/volumes"
Jul 28 17:43:37.581013 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:37.580984    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:43:37.581537 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:37.581506    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 28 17:43:47.582341 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:47.582300    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 28 17:43:57.581988 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:43:57.581952    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 28 17:44:07.582368 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:07.582327    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 28 17:44:17.582881 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:17.582836    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:44:38.465047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.465013    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"]
Jul 28 17:44:38.465571 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.465557    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container"
Jul 28 17:44:38.465619 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.465584    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container"
Jul 28 17:44:38.465659 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.465622    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kube-rbac-proxy"
Jul 28 17:44:38.465659 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.465631    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kube-rbac-proxy"
Jul 28 17:44:38.465734 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.465728    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kube-rbac-proxy"
Jul 28 17:44:38.465769 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.465743    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="040fce8d-7c1d-4b53-b4af-e6ddc6ebd1e9" containerName="kserve-container"
Jul 28 17:44:38.469383 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.469365    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.472632 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.472614    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-61a23-predictor-serving-cert\""
Jul 28 17:44:38.472949 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.472930    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-61a23-kube-rbac-proxy-sar-config\""
Jul 28 17:44:38.494407 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.494383    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"]
Jul 28 17:44:38.553294 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.553256    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.553455 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.553401    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ab8bb64-52b0-41a9-909c-b0b41258bea5-proxy-tls\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.553455 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.553440    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4xvr\" (UniqueName: \"kubernetes.io/projected/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-api-access-d4xvr\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.571091 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.571064    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"]
Jul 28 17:44:38.571306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.571284    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container" containerID="cri-o://1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec" gracePeriod=30
Jul 28 17:44:38.571384 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.571307    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kube-rbac-proxy" containerID="cri-o://8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a" gracePeriod=30
Jul 28 17:44:38.654424 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.654390    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ab8bb64-52b0-41a9-909c-b0b41258bea5-proxy-tls\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.654618 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.654437    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4xvr\" (UniqueName: \"kubernetes.io/projected/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-api-access-d4xvr\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.654698 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.654669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.655388 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.655364    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.656824 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.656796    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ab8bb64-52b0-41a9-909c-b0b41258bea5-proxy-tls\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.666761 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.666739    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4xvr\" (UniqueName: \"kubernetes.io/projected/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-api-access-d4xvr\") pod \"error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") " pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.780709 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.780618    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:38.905042 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.905003    2556 generic.go:356] "Generic (PLEG): container finished" podID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerID="8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a" exitCode=2
Jul 28 17:44:38.905217 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.905075    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" event={"ID":"74662a03-b254-4f2c-ac7c-d5d7328879e3","Type":"ContainerDied","Data":"8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a"}
Jul 28 17:44:38.964085 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:38.964060    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"]
Jul 28 17:44:38.966716 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:44:38.966686    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab8bb64_52b0_41a9_909c_b0b41258bea5.slice/crio-bb8646e62e9a17bfffdd5d70b5d11b4ef9f2e4033e5958017395457dcd211316 WatchSource:0}: Error finding container bb8646e62e9a17bfffdd5d70b5d11b4ef9f2e4033e5958017395457dcd211316: Status 404 returned error can't find the container with id bb8646e62e9a17bfffdd5d70b5d11b4ef9f2e4033e5958017395457dcd211316
Jul 28 17:44:39.273399 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.273360    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"]
Jul 28 17:44:39.277655 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.277631    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.283190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.282838    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-614a5-predictor-serving-cert\""
Jul 28 17:44:39.283341 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.283182    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-614a5-kube-rbac-proxy-sar-config\""
Jul 28 17:44:39.329926 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.329858    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"]
Jul 28 17:44:39.330132 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.330104    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container" containerID="cri-o://53e2c5dcfaa77c134ce78e349c41c4f86a12649e3886bfe332712e928431204e" gracePeriod=30
Jul 28 17:44:39.330214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.330165    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kube-rbac-proxy" containerID="cri-o://f7c123c5b8d9b64b9e4b4c7bc72bf9c76cb094e9334841dfe65f4db51bfbaf7c" gracePeriod=30
Jul 28 17:44:39.351364 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.351337    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"]
Jul 28 17:44:39.360386 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.360362    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gm54\" (UniqueName: \"kubernetes.io/projected/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-api-access-2gm54\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.360504 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.360410    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dd18072-0f56-4a5b-8bfb-d06423530a14-proxy-tls\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.360570 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.360518    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.461946 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.461909    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gm54\" (UniqueName: \"kubernetes.io/projected/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-api-access-2gm54\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.462129 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.461959    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dd18072-0f56-4a5b-8bfb-d06423530a14-proxy-tls\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.462129 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.461983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.462645 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.462625    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.464240 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.464219    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dd18072-0f56-4a5b-8bfb-d06423530a14-proxy-tls\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.472061 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.472044    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gm54\" (UniqueName: \"kubernetes.io/projected/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-api-access-2gm54\") pod \"error-404-isvc-614a5-predictor-55c87cb68d-g9s48\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") " pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.592710 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.592679    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.750634 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.750607    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"]
Jul 28 17:44:39.753190 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:44:39.753165    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd18072_0f56_4a5b_8bfb_d06423530a14.slice/crio-f645553dcb8c8e9db15441d3f2fd53a0e169b8214d875c69a3e0c0ed08ba99fb WatchSource:0}: Error finding container f645553dcb8c8e9db15441d3f2fd53a0e169b8214d875c69a3e0c0ed08ba99fb: Status 404 returned error can't find the container with id f645553dcb8c8e9db15441d3f2fd53a0e169b8214d875c69a3e0c0ed08ba99fb
Jul 28 17:44:39.911206 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.911171    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" event={"ID":"2ab8bb64-52b0-41a9-909c-b0b41258bea5","Type":"ContainerStarted","Data":"3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139"}
Jul 28 17:44:39.911390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.911243    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" event={"ID":"2ab8bb64-52b0-41a9-909c-b0b41258bea5","Type":"ContainerStarted","Data":"ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea"}
Jul 28 17:44:39.911390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.911277    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" event={"ID":"2ab8bb64-52b0-41a9-909c-b0b41258bea5","Type":"ContainerStarted","Data":"bb8646e62e9a17bfffdd5d70b5d11b4ef9f2e4033e5958017395457dcd211316"}
Jul 28 17:44:39.911390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.911352    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:39.911603 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.911402    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:39.913056 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.913026    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Jul 28 17:44:39.913185 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.913078    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" event={"ID":"8dd18072-0f56-4a5b-8bfb-d06423530a14","Type":"ContainerStarted","Data":"4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4"}
Jul 28 17:44:39.913185 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.913125    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" event={"ID":"8dd18072-0f56-4a5b-8bfb-d06423530a14","Type":"ContainerStarted","Data":"910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b"}
Jul 28 17:44:39.913185 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.913157    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" event={"ID":"8dd18072-0f56-4a5b-8bfb-d06423530a14","Type":"ContainerStarted","Data":"f645553dcb8c8e9db15441d3f2fd53a0e169b8214d875c69a3e0c0ed08ba99fb"}
Jul 28 17:44:39.913350 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.913192    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.913350 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.913212    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:39.914484 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.914457    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Jul 28 17:44:39.915010 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.914984    2556 generic.go:356] "Generic (PLEG): container finished" podID="0abeb40e-8792-44da-a994-2ae33123547e" containerID="f7c123c5b8d9b64b9e4b4c7bc72bf9c76cb094e9334841dfe65f4db51bfbaf7c" exitCode=2
Jul 28 17:44:39.915122 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.915061    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" event={"ID":"0abeb40e-8792-44da-a994-2ae33123547e","Type":"ContainerDied","Data":"f7c123c5b8d9b64b9e4b4c7bc72bf9c76cb094e9334841dfe65f4db51bfbaf7c"}
Jul 28 17:44:39.935232 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.935194    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podStartSLOduration=1.935180371 podStartE2EDuration="1.935180371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:44:39.933594091 +0000 UTC m=+1301.879333275" watchObservedRunningTime="2026-07-28 17:44:39.935180371 +0000 UTC m=+1301.880919534"
Jul 28 17:44:39.959971 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:39.959928    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podStartSLOduration=0.959915873 podStartE2EDuration="959.915873ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:44:39.958100078 +0000 UTC m=+1301.903839241" watchObservedRunningTime="2026-07-28 17:44:39.959915873 +0000 UTC m=+1301.905655036"
Jul 28 17:44:40.920483 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:40.920429    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Jul 28 17:44:40.920992 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:40.920603    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Jul 28 17:44:42.130276 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.130253    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:44:42.188027 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.187992    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-rbac-proxy-sar-config\") pod \"74662a03-b254-4f2c-ac7c-d5d7328879e3\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") "
Jul 28 17:44:42.188236 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.188097    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7smm\" (UniqueName: \"kubernetes.io/projected/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-api-access-j7smm\") pod \"74662a03-b254-4f2c-ac7c-d5d7328879e3\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") "
Jul 28 17:44:42.188236 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.188205    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls\") pod \"74662a03-b254-4f2c-ac7c-d5d7328879e3\" (UID: \"74662a03-b254-4f2c-ac7c-d5d7328879e3\") "
Jul 28 17:44:42.188890 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.188844    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "74662a03-b254-4f2c-ac7c-d5d7328879e3" (UID: "74662a03-b254-4f2c-ac7c-d5d7328879e3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:44:42.191444 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.191411    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-api-access-j7smm" (OuterVolumeSpecName: "kube-api-access-j7smm") pod "74662a03-b254-4f2c-ac7c-d5d7328879e3" (UID: "74662a03-b254-4f2c-ac7c-d5d7328879e3"). InnerVolumeSpecName "kube-api-access-j7smm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:44:42.192088 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.192059    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "74662a03-b254-4f2c-ac7c-d5d7328879e3" (UID: "74662a03-b254-4f2c-ac7c-d5d7328879e3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:44:42.289461 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.289426    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/74662a03-b254-4f2c-ac7c-d5d7328879e3-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:44:42.289461 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.289455    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:44:42.289461 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.289464    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7smm\" (UniqueName: \"kubernetes.io/projected/74662a03-b254-4f2c-ac7c-d5d7328879e3-kube-api-access-j7smm\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:44:42.578251 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.578159    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.41:8643/healthz\": dial tcp 10.134.0.41:8643: connect: connection refused"
Jul 28 17:44:42.931308 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.931271    2556 generic.go:356] "Generic (PLEG): container finished" podID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerID="1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec" exitCode=0
Jul 28 17:44:42.931501 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.931311    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" event={"ID":"74662a03-b254-4f2c-ac7c-d5d7328879e3","Type":"ContainerDied","Data":"1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec"}
Jul 28 17:44:42.931501 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.931370    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw" event={"ID":"74662a03-b254-4f2c-ac7c-d5d7328879e3","Type":"ContainerDied","Data":"292e12f2449694c28c42bc19253c77ffe2b8dc3305249227595dcb231db2c648"}
Jul 28 17:44:42.931501 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.931386    2556 scope.go:117] "RemoveContainer" containerID="8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a"
Jul 28 17:44:42.931501 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.931320    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"
Jul 28 17:44:42.933346 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.933320    2556 generic.go:356] "Generic (PLEG): container finished" podID="0abeb40e-8792-44da-a994-2ae33123547e" containerID="53e2c5dcfaa77c134ce78e349c41c4f86a12649e3886bfe332712e928431204e" exitCode=0
Jul 28 17:44:42.933463 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.933404    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" event={"ID":"0abeb40e-8792-44da-a994-2ae33123547e","Type":"ContainerDied","Data":"53e2c5dcfaa77c134ce78e349c41c4f86a12649e3886bfe332712e928431204e"}
Jul 28 17:44:42.939759 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.939743    2556 scope.go:117] "RemoveContainer" containerID="1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec"
Jul 28 17:44:42.947186 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.947158    2556 scope.go:117] "RemoveContainer" containerID="8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a"
Jul 28 17:44:42.947451 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:44:42.947432    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a\": container with ID starting with 8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a not found: ID does not exist" containerID="8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a"
Jul 28 17:44:42.947498 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.947462    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a"} err="failed to get container status \"8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a\": rpc error: code = NotFound desc = could not find container \"8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a\": container with ID starting with 8d571e145656a18986fde2ecb496fb9e5bbe48fa3d4eed7bb4b49b10f655dd9a not found: ID does not exist"
Jul 28 17:44:42.947498 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.947481    2556 scope.go:117] "RemoveContainer" containerID="1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec"
Jul 28 17:44:42.947715 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:44:42.947696    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec\": container with ID starting with 1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec not found: ID does not exist" containerID="1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec"
Jul 28 17:44:42.947759 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.947720    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec"} err="failed to get container status \"1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec\": rpc error: code = NotFound desc = could not find container \"1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec\": container with ID starting with 1da331f458c1703374e7bb732902ec18d52dc0a921d7e5c41a14a419814679ec not found: ID does not exist"
Jul 28 17:44:42.950574 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.950550    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"]
Jul 28 17:44:42.954642 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:42.954621    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b1dfe-predictor-54cdc9876b-kc6bw"]
Jul 28 17:44:43.187345 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.187317    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:44:43.300362 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.300332    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abeb40e-8792-44da-a994-2ae33123547e-kube-rbac-proxy-sar-config\") pod \"0abeb40e-8792-44da-a994-2ae33123547e\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") "
Jul 28 17:44:43.300560 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.300402    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls\") pod \"0abeb40e-8792-44da-a994-2ae33123547e\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") "
Jul 28 17:44:43.300560 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.300421    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t54l9\" (UniqueName: \"kubernetes.io/projected/0abeb40e-8792-44da-a994-2ae33123547e-kube-api-access-t54l9\") pod \"0abeb40e-8792-44da-a994-2ae33123547e\" (UID: \"0abeb40e-8792-44da-a994-2ae33123547e\") "
Jul 28 17:44:43.300765 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.300733    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0abeb40e-8792-44da-a994-2ae33123547e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0abeb40e-8792-44da-a994-2ae33123547e" (UID: "0abeb40e-8792-44da-a994-2ae33123547e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:44:43.302557 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.302533    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0abeb40e-8792-44da-a994-2ae33123547e" (UID: "0abeb40e-8792-44da-a994-2ae33123547e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:44:43.302557 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.302545    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0abeb40e-8792-44da-a994-2ae33123547e-kube-api-access-t54l9" (OuterVolumeSpecName: "kube-api-access-t54l9") pod "0abeb40e-8792-44da-a994-2ae33123547e" (UID: "0abeb40e-8792-44da-a994-2ae33123547e"). InnerVolumeSpecName "kube-api-access-t54l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:44:43.401863 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.401828    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abeb40e-8792-44da-a994-2ae33123547e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:44:43.401863 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.401857    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abeb40e-8792-44da-a994-2ae33123547e-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:44:43.401863 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.401891    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t54l9\" (UniqueName: \"kubernetes.io/projected/0abeb40e-8792-44da-a994-2ae33123547e-kube-api-access-t54l9\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:44:43.940120 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.940089    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"
Jul 28 17:44:43.940326 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.940121    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5" event={"ID":"0abeb40e-8792-44da-a994-2ae33123547e","Type":"ContainerDied","Data":"b489f2664601c18875e72eebea9db96ff1f7fc3e210a88719da54a7a6dbe7d82"}
Jul 28 17:44:43.940326 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.940197    2556 scope.go:117] "RemoveContainer" containerID="f7c123c5b8d9b64b9e4b4c7bc72bf9c76cb094e9334841dfe65f4db51bfbaf7c"
Jul 28 17:44:43.949080 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.949061    2556 scope.go:117] "RemoveContainer" containerID="53e2c5dcfaa77c134ce78e349c41c4f86a12649e3886bfe332712e928431204e"
Jul 28 17:44:43.961699 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.961676    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"]
Jul 28 17:44:43.966188 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:43.966166    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8b8b5-predictor-68f9fbd6b8-5djw5"]
Jul 28 17:44:44.597214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:44.594312    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0abeb40e-8792-44da-a994-2ae33123547e" path="/var/lib/kubelet/pods/0abeb40e-8792-44da-a994-2ae33123547e/volumes"
Jul 28 17:44:44.597214 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:44.595131    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" path="/var/lib/kubelet/pods/74662a03-b254-4f2c-ac7c-d5d7328879e3/volumes"
Jul 28 17:44:45.924494 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:45.924466    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:44:45.925028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:45.924998    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Jul 28 17:44:45.926006 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:45.925941    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:44:45.926458 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:45.926437    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Jul 28 17:44:55.925046 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:55.925007    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Jul 28 17:44:55.927310 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:44:55.927284    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Jul 28 17:45:05.925768 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:05.925728    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Jul 28 17:45:05.926968 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:05.926945    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Jul 28 17:45:15.925724 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:15.925684    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Jul 28 17:45:15.926919 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:15.926896    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Jul 28 17:45:25.926115 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:25.926031    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:45:25.927223 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:25.927208    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:45:52.600056 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.600021    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"]
Jul 28 17:45:52.600655 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.600286    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container" containerID="cri-o://ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea" gracePeriod=30
Jul 28 17:45:52.600655 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.600583    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kube-rbac-proxy" containerID="cri-o://3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139" gracePeriod=30
Jul 28 17:45:52.643291 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643260    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"]
Jul 28 17:45:52.643635 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643624    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container"
Jul 28 17:45:52.643683 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643637    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container"
Jul 28 17:45:52.643683 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643659    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kube-rbac-proxy"
Jul 28 17:45:52.643683 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643664    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kube-rbac-proxy"
Jul 28 17:45:52.643683 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643671    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container"
Jul 28 17:45:52.643683 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643676    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container"
Jul 28 17:45:52.643683 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643685    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kube-rbac-proxy"
Jul 28 17:45:52.643859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643690    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kube-rbac-proxy"
Jul 28 17:45:52.643859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643749    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kube-rbac-proxy"
Jul 28 17:45:52.643859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643758    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kserve-container"
Jul 28 17:45:52.643859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643768    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="74662a03-b254-4f2c-ac7c-d5d7328879e3" containerName="kube-rbac-proxy"
Jul 28 17:45:52.643859 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.643773    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0abeb40e-8792-44da-a994-2ae33123547e" containerName="kserve-container"
Jul 28 17:45:52.647257 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.647238    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.651164 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.651143    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-787a1-predictor-serving-cert\""
Jul 28 17:45:52.651254 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.651190    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-787a1-kube-rbac-proxy-sar-config\""
Jul 28 17:45:52.661274 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.661252    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"]
Jul 28 17:45:52.721361 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.721335    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/784170c7-f416-4d44-8c2d-ee625229e312-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.721558 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.721497    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/784170c7-f416-4d44-8c2d-ee625229e312-proxy-tls\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.721558 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.721539    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8dp4\" (UniqueName: \"kubernetes.io/projected/784170c7-f416-4d44-8c2d-ee625229e312-kube-api-access-b8dp4\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.822103 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.822072    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/784170c7-f416-4d44-8c2d-ee625229e312-proxy-tls\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.822278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.822115    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8dp4\" (UniqueName: \"kubernetes.io/projected/784170c7-f416-4d44-8c2d-ee625229e312-kube-api-access-b8dp4\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.822278 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.822169    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/784170c7-f416-4d44-8c2d-ee625229e312-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.822865 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.822846    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/784170c7-f416-4d44-8c2d-ee625229e312-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.824554 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.824525    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/784170c7-f416-4d44-8c2d-ee625229e312-proxy-tls\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.835202 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.835180    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8dp4\" (UniqueName: \"kubernetes.io/projected/784170c7-f416-4d44-8c2d-ee625229e312-kube-api-access-b8dp4\") pod \"error-404-isvc-787a1-predictor-fcb464fdb-zzl47\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") " pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:52.957850 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:52.957819    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:53.108057 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.107776    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"]
Jul 28 17:45:53.110813 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:45:53.110787    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784170c7_f416_4d44_8c2d_ee625229e312.slice/crio-23f2b18dded1f2e8a18723223c970493b620b0b2ff838f11670a5f055d1fdfaa WatchSource:0}: Error finding container 23f2b18dded1f2e8a18723223c970493b620b0b2ff838f11670a5f055d1fdfaa: Status 404 returned error can't find the container with id 23f2b18dded1f2e8a18723223c970493b620b0b2ff838f11670a5f055d1fdfaa
Jul 28 17:45:53.269317 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.269280    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" event={"ID":"784170c7-f416-4d44-8c2d-ee625229e312","Type":"ContainerStarted","Data":"eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa"}
Jul 28 17:45:53.269533 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.269512    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:53.269625 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.269569    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" event={"ID":"784170c7-f416-4d44-8c2d-ee625229e312","Type":"ContainerStarted","Data":"ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f"}
Jul 28 17:45:53.269625 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.269601    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" event={"ID":"784170c7-f416-4d44-8c2d-ee625229e312","Type":"ContainerStarted","Data":"23f2b18dded1f2e8a18723223c970493b620b0b2ff838f11670a5f055d1fdfaa"}
Jul 28 17:45:53.269737 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.269630    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:53.270606 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.270575    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Jul 28 17:45:53.271236 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.271208    2556 generic.go:356] "Generic (PLEG): container finished" podID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerID="3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139" exitCode=2
Jul 28 17:45:53.271315 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.271247    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" event={"ID":"2ab8bb64-52b0-41a9-909c-b0b41258bea5","Type":"ContainerDied","Data":"3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139"}
Jul 28 17:45:53.291173 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:53.291117    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podStartSLOduration=1.291102062 podStartE2EDuration="1.291102062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:45:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:45:53.289887311 +0000 UTC m=+1375.235626477" watchObservedRunningTime="2026-07-28 17:45:53.291102062 +0000 UTC m=+1375.236841226"
Jul 28 17:45:54.276128 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:54.276093    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Jul 28 17:45:55.652468 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.652445    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:45:55.744270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.744194    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-rbac-proxy-sar-config\") pod \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") "
Jul 28 17:45:55.744270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.744231    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ab8bb64-52b0-41a9-909c-b0b41258bea5-proxy-tls\") pod \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") "
Jul 28 17:45:55.744270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.744269    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4xvr\" (UniqueName: \"kubernetes.io/projected/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-api-access-d4xvr\") pod \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\" (UID: \"2ab8bb64-52b0-41a9-909c-b0b41258bea5\") "
Jul 28 17:45:55.744541 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.744521    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2ab8bb64-52b0-41a9-909c-b0b41258bea5" (UID: "2ab8bb64-52b0-41a9-909c-b0b41258bea5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:45:55.746336 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.746311    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-api-access-d4xvr" (OuterVolumeSpecName: "kube-api-access-d4xvr") pod "2ab8bb64-52b0-41a9-909c-b0b41258bea5" (UID: "2ab8bb64-52b0-41a9-909c-b0b41258bea5"). InnerVolumeSpecName "kube-api-access-d4xvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:45:55.746452 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.746367    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ab8bb64-52b0-41a9-909c-b0b41258bea5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2ab8bb64-52b0-41a9-909c-b0b41258bea5" (UID: "2ab8bb64-52b0-41a9-909c-b0b41258bea5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:45:55.845132 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.845102    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4xvr\" (UniqueName: \"kubernetes.io/projected/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-api-access-d4xvr\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:45:55.845132 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.845127    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ab8bb64-52b0-41a9-909c-b0b41258bea5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:45:55.845132 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:55.845137    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ab8bb64-52b0-41a9-909c-b0b41258bea5-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:45:56.291173 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.291135    2556 generic.go:356] "Generic (PLEG): container finished" podID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerID="ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea" exitCode=0
Jul 28 17:45:56.291378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.291187    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"
Jul 28 17:45:56.291378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.291226    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" event={"ID":"2ab8bb64-52b0-41a9-909c-b0b41258bea5","Type":"ContainerDied","Data":"ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea"}
Jul 28 17:45:56.291378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.291300    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w" event={"ID":"2ab8bb64-52b0-41a9-909c-b0b41258bea5","Type":"ContainerDied","Data":"bb8646e62e9a17bfffdd5d70b5d11b4ef9f2e4033e5958017395457dcd211316"}
Jul 28 17:45:56.291378 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.291337    2556 scope.go:117] "RemoveContainer" containerID="3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139"
Jul 28 17:45:56.300257 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.300241    2556 scope.go:117] "RemoveContainer" containerID="ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea"
Jul 28 17:45:56.308410 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.308389    2556 scope.go:117] "RemoveContainer" containerID="3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139"
Jul 28 17:45:56.308647 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:45:56.308630    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139\": container with ID starting with 3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139 not found: ID does not exist" containerID="3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139"
Jul 28 17:45:56.308702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.308657    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139"} err="failed to get container status \"3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139\": rpc error: code = NotFound desc = could not find container \"3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139\": container with ID starting with 3d99d054b47864c5f84206944d2703040a7c627ff60ac66789dc9c80d659e139 not found: ID does not exist"
Jul 28 17:45:56.308702 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.308676    2556 scope.go:117] "RemoveContainer" containerID="ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea"
Jul 28 17:45:56.308931 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:45:56.308911    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea\": container with ID starting with ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea not found: ID does not exist" containerID="ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea"
Jul 28 17:45:56.308985 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.308936    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea"} err="failed to get container status \"ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea\": rpc error: code = NotFound desc = could not find container \"ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea\": container with ID starting with ee55fbd5edc11aa3aac0b63817e717c8b5b92e2fb4931d5e04699873a3ee85ea not found: ID does not exist"
Jul 28 17:45:56.314034 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.314012    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"]
Jul 28 17:45:56.317784 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.317764    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-61a23-predictor-fd5b4c6d4-52d4w"]
Jul 28 17:45:56.591285 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:56.591210    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" path="/var/lib/kubelet/pods/2ab8bb64-52b0-41a9-909c-b0b41258bea5/volumes"
Jul 28 17:45:59.280811 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:59.280783    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:45:59.281298 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:45:59.281269    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Jul 28 17:46:09.281528 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:46:09.281487    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Jul 28 17:46:19.282031 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:46:19.281985    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Jul 28 17:46:29.281430 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:46:29.281392    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Jul 28 17:46:39.282043 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:46:39.282012    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:47:58.553553 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:47:58.553522    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:47:58.555960 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:47:58.555935    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:52:58.587190 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:52:58.587158    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:52:58.590566 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:52:58.590544    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:53:53.745681 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.745635    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"]
Jul 28 17:53:53.746224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.746096    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container"
Jul 28 17:53:53.746224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.746111    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container"
Jul 28 17:53:53.746224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.746126    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kube-rbac-proxy"
Jul 28 17:53:53.746224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.746131    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kube-rbac-proxy"
Jul 28 17:53:53.746224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.746197    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kserve-container"
Jul 28 17:53:53.746224 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.746206    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab8bb64-52b0-41a9-909c-b0b41258bea5" containerName="kube-rbac-proxy"
Jul 28 17:53:53.749437 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.749416    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.752426 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.752407    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f7704-predictor-serving-cert\""
Jul 28 17:53:53.753924 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.753901    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f7704-kube-rbac-proxy-sar-config\""
Jul 28 17:53:53.762384 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.762360    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"]
Jul 28 17:53:53.810391 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.810361    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"]
Jul 28 17:53:53.810626 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.810598    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" containerID="cri-o://910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b" gracePeriod=30
Jul 28 17:53:53.810742 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.810619    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kube-rbac-proxy" containerID="cri-o://4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4" gracePeriod=30
Jul 28 17:53:53.853975 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.853936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.854158 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.853982    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b54m9\" (UniqueName: \"kubernetes.io/projected/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-api-access-b54m9\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.854158 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.854019    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.955161 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.955131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.955381 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.955181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b54m9\" (UniqueName: \"kubernetes.io/projected/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-api-access-b54m9\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.955381 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.955205    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.955494 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:53:53.955376    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-f7704-predictor-serving-cert: secret "error-404-isvc-f7704-predictor-serving-cert" not found
Jul 28 17:53:53.955494 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:53:53.955450    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls podName:8e664fad-d9a9-4cb8-914e-aa23c663d7aa nodeName:}" failed. No retries permitted until 2026-07-28 17:53:54.455427671 +0000 UTC m=+1856.401166816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls") pod "error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" (UID: "8e664fad-d9a9-4cb8-914e-aa23c663d7aa") : secret "error-404-isvc-f7704-predictor-serving-cert" not found
Jul 28 17:53:53.955994 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.955976    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:53.963625 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:53.963594    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b54m9\" (UniqueName: \"kubernetes.io/projected/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-api-access-b54m9\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:54.460296 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:54.460260    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:54.462765 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:54.462745    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls\") pod \"error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") " pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:54.541606 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:54.541563    2556 generic.go:356] "Generic (PLEG): container finished" podID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerID="4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4" exitCode=2
Jul 28 17:53:54.541819 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:54.541676    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" event={"ID":"8dd18072-0f56-4a5b-8bfb-d06423530a14","Type":"ContainerDied","Data":"4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4"}
Jul 28 17:53:54.661406 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:54.661370    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:54.808435 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:54.808406    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"]
Jul 28 17:53:54.810195 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:53:54.810165    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e664fad_d9a9_4cb8_914e_aa23c663d7aa.slice/crio-c195635c4bdf67cba5b67df75a950f2e8d388dd355fbd3c1a35c93b55ec66b81 WatchSource:0}: Error finding container c195635c4bdf67cba5b67df75a950f2e8d388dd355fbd3c1a35c93b55ec66b81: Status 404 returned error can't find the container with id c195635c4bdf67cba5b67df75a950f2e8d388dd355fbd3c1a35c93b55ec66b81
Jul 28 17:53:55.548402 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:55.548366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" event={"ID":"8e664fad-d9a9-4cb8-914e-aa23c663d7aa","Type":"ContainerStarted","Data":"6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7"}
Jul 28 17:53:55.548625 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:55.548430    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" event={"ID":"8e664fad-d9a9-4cb8-914e-aa23c663d7aa","Type":"ContainerStarted","Data":"656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7"}
Jul 28 17:53:55.548625 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:55.548457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" event={"ID":"8e664fad-d9a9-4cb8-914e-aa23c663d7aa","Type":"ContainerStarted","Data":"c195635c4bdf67cba5b67df75a950f2e8d388dd355fbd3c1a35c93b55ec66b81"}
Jul 28 17:53:55.548625 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:55.548487    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:55.569375 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:55.569321    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podStartSLOduration=2.569305005 podStartE2EDuration="2.569305005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:53:55.567603856 +0000 UTC m=+1857.513343020" watchObservedRunningTime="2026-07-28 17:53:55.569305005 +0000 UTC m=+1857.515044168"
Jul 28 17:53:55.920785 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:55.920688    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.43:8643/healthz\": dial tcp 10.134.0.43:8643: connect: connection refused"
Jul 28 17:53:55.925828 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:55.925803    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Jul 28 17:53:56.553356 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:56.553323    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:53:56.554752 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:56.554726    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Jul 28 17:53:57.068971 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.068943    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:53:57.181844 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.181819    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dd18072-0f56-4a5b-8bfb-d06423530a14-proxy-tls\") pod \"8dd18072-0f56-4a5b-8bfb-d06423530a14\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") "
Jul 28 17:53:57.182069 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.181929    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gm54\" (UniqueName: \"kubernetes.io/projected/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-api-access-2gm54\") pod \"8dd18072-0f56-4a5b-8bfb-d06423530a14\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") "
Jul 28 17:53:57.182069 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.181982    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-rbac-proxy-sar-config\") pod \"8dd18072-0f56-4a5b-8bfb-d06423530a14\" (UID: \"8dd18072-0f56-4a5b-8bfb-d06423530a14\") "
Jul 28 17:53:57.182370 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.182346    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8dd18072-0f56-4a5b-8bfb-d06423530a14" (UID: "8dd18072-0f56-4a5b-8bfb-d06423530a14"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:53:57.184002 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.183980    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd18072-0f56-4a5b-8bfb-d06423530a14-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8dd18072-0f56-4a5b-8bfb-d06423530a14" (UID: "8dd18072-0f56-4a5b-8bfb-d06423530a14"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:53:57.184108 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.184095    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-api-access-2gm54" (OuterVolumeSpecName: "kube-api-access-2gm54") pod "8dd18072-0f56-4a5b-8bfb-d06423530a14" (UID: "8dd18072-0f56-4a5b-8bfb-d06423530a14"). InnerVolumeSpecName "kube-api-access-2gm54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:53:57.283614 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.283572    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gm54\" (UniqueName: \"kubernetes.io/projected/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-api-access-2gm54\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:53:57.283614 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.283604    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dd18072-0f56-4a5b-8bfb-d06423530a14-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:53:57.283614 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.283614    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dd18072-0f56-4a5b-8bfb-d06423530a14-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:53:57.560119 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.560037    2556 generic.go:356] "Generic (PLEG): container finished" podID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerID="910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b" exitCode=0
Jul 28 17:53:57.560119 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.560085    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"
Jul 28 17:53:57.560306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.560124    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" event={"ID":"8dd18072-0f56-4a5b-8bfb-d06423530a14","Type":"ContainerDied","Data":"910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b"}
Jul 28 17:53:57.560306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.560184    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48" event={"ID":"8dd18072-0f56-4a5b-8bfb-d06423530a14","Type":"ContainerDied","Data":"f645553dcb8c8e9db15441d3f2fd53a0e169b8214d875c69a3e0c0ed08ba99fb"}
Jul 28 17:53:57.560306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.560209    2556 scope.go:117] "RemoveContainer" containerID="4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4"
Jul 28 17:53:57.560819 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.560786    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Jul 28 17:53:57.569883 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.569852    2556 scope.go:117] "RemoveContainer" containerID="910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b"
Jul 28 17:53:57.578183 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.578166    2556 scope.go:117] "RemoveContainer" containerID="4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4"
Jul 28 17:53:57.578468 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:53:57.578447    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4\": container with ID starting with 4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4 not found: ID does not exist" containerID="4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4"
Jul 28 17:53:57.578538 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.578477    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4"} err="failed to get container status \"4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4\": rpc error: code = NotFound desc = could not find container \"4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4\": container with ID starting with 4c4a03d59d0f9cc4ad0b5e552369db5df171a847f43d71dcbeea0d683a86b2a4 not found: ID does not exist"
Jul 28 17:53:57.578538 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.578497    2556 scope.go:117] "RemoveContainer" containerID="910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b"
Jul 28 17:53:57.578718 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:53:57.578704    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b\": container with ID starting with 910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b not found: ID does not exist" containerID="910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b"
Jul 28 17:53:57.578763 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.578720    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b"} err="failed to get container status \"910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b\": rpc error: code = NotFound desc = could not find container \"910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b\": container with ID starting with 910e692336c0c679a41886ab5b65e9f40a894176306516b1e5b083dcf2bd426b not found: ID does not exist"
Jul 28 17:53:57.588969 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.588927    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"]
Jul 28 17:53:57.602462 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:57.602431    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-614a5-predictor-55c87cb68d-g9s48"]
Jul 28 17:53:58.590409 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:53:58.590373    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" path="/var/lib/kubelet/pods/8dd18072-0f56-4a5b-8bfb-d06423530a14/volumes"
Jul 28 17:54:02.565156 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:54:02.565127    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:54:02.565627 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:54:02.565540    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Jul 28 17:54:12.565685 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:54:12.565647    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Jul 28 17:54:22.566362 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:54:22.566276    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Jul 28 17:54:32.566096 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:54:32.566055    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Jul 28 17:54:42.567047 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:54:42.567012    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:55:04.183576 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.183541    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"]
Jul 28 17:55:04.184028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.183764    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container" containerID="cri-o://656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7" gracePeriod=30
Jul 28 17:55:04.184028 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.183798    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kube-rbac-proxy" containerID="cri-o://6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7" gracePeriod=30
Jul 28 17:55:04.204184 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.204157    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"]
Jul 28 17:55:04.204558 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.204547    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container"
Jul 28 17:55:04.204602 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.204561    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container"
Jul 28 17:55:04.204602 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.204571    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kube-rbac-proxy"
Jul 28 17:55:04.204602 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.204577    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kube-rbac-proxy"
Jul 28 17:55:04.204699 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.204635    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kserve-container"
Jul 28 17:55:04.204699 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.204647    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd18072-0f56-4a5b-8bfb-d06423530a14" containerName="kube-rbac-proxy"
Jul 28 17:55:04.210015 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.209994    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.215080 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.215058    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7d999-predictor-serving-cert\""
Jul 28 17:55:04.215206 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.215165    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7d999-kube-rbac-proxy-sar-config\""
Jul 28 17:55:04.222285 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.222260    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"]
Jul 28 17:55:04.290287 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.290254    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqsjg\" (UniqueName: \"kubernetes.io/projected/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-api-access-gqsjg\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.290472 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.290322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.290472 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.290433    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.391903 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.391837    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.392106 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.391949    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqsjg\" (UniqueName: \"kubernetes.io/projected/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-api-access-gqsjg\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.392106 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.392010    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.392238 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:55:04.392162    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-7d999-predictor-serving-cert: secret "error-404-isvc-7d999-predictor-serving-cert" not found
Jul 28 17:55:04.392293 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:55:04.392243    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls podName:e53e30d3-5717-4f68-b3c4-80fe2ae9be48 nodeName:}" failed. No retries permitted until 2026-07-28 17:55:04.892221194 +0000 UTC m=+1926.837960353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls") pod "error-404-isvc-7d999-predictor-566585ff9d-rtwzr" (UID: "e53e30d3-5717-4f68-b3c4-80fe2ae9be48") : secret "error-404-isvc-7d999-predictor-serving-cert" not found
Jul 28 17:55:04.392492 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.392469    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.402360 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.402329    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqsjg\" (UniqueName: \"kubernetes.io/projected/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-api-access-gqsjg\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.882800 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.882765    2556 generic.go:356] "Generic (PLEG): container finished" podID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerID="6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7" exitCode=2
Jul 28 17:55:04.882989 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.882836    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" event={"ID":"8e664fad-d9a9-4cb8-914e-aa23c663d7aa","Type":"ContainerDied","Data":"6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7"}
Jul 28 17:55:04.895490 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.895467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:04.897790 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:04.897765    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls\") pod \"error-404-isvc-7d999-predictor-566585ff9d-rtwzr\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") " pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:05.122455 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:05.122419    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:05.262781 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:05.262749    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"]
Jul 28 17:55:05.266440 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:55:05.266408    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53e30d3_5717_4f68_b3c4_80fe2ae9be48.slice/crio-af6944e027f27dc0452576b7125cc499e78e6c06b4af9cb7e11613350f171dbc WatchSource:0}: Error finding container af6944e027f27dc0452576b7125cc499e78e6c06b4af9cb7e11613350f171dbc: Status 404 returned error can't find the container with id af6944e027f27dc0452576b7125cc499e78e6c06b4af9cb7e11613350f171dbc
Jul 28 17:55:05.891819 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:05.891774    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" event={"ID":"e53e30d3-5717-4f68-b3c4-80fe2ae9be48","Type":"ContainerStarted","Data":"baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510"}
Jul 28 17:55:05.892049 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:05.891860    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" event={"ID":"e53e30d3-5717-4f68-b3c4-80fe2ae9be48","Type":"ContainerStarted","Data":"5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496"}
Jul 28 17:55:05.892049 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:05.891917    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" event={"ID":"e53e30d3-5717-4f68-b3c4-80fe2ae9be48","Type":"ContainerStarted","Data":"af6944e027f27dc0452576b7125cc499e78e6c06b4af9cb7e11613350f171dbc"}
Jul 28 17:55:05.892049 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:05.891999    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:05.914129 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:05.914080    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podStartSLOduration=1.914063244 podStartE2EDuration="1.914063244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:55:05.91217003 +0000 UTC m=+1927.857909193" watchObservedRunningTime="2026-07-28 17:55:05.914063244 +0000 UTC m=+1927.859802387"
Jul 28 17:55:06.897589 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:06.897559    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:06.898665 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:06.898637    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Jul 28 17:55:07.531008 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.530969    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"]
Jul 28 17:55:07.536722 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.536697    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.539186 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.539146    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-cb1ee-predictor-serving-cert\""
Jul 28 17:55:07.539186 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.539178    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-cb1ee-kube-rbac-proxy-sar-config\""
Jul 28 17:55:07.546774 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.546744    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"]
Jul 28 17:55:07.576969 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.576931    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:55:07.592628 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.592588    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"]
Jul 28 17:55:07.592834 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.592813    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" containerID="cri-o://ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f" gracePeriod=30
Jul 28 17:55:07.592953 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.592927    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kube-rbac-proxy" containerID="cri-o://eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa" gracePeriod=30
Jul 28 17:55:07.631815 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.631772    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.632139 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.631933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c10a7e1-e62e-4167-b3b4-41d398b449ba-proxy-tls\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.632139 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.631985    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnnp4\" (UniqueName: \"kubernetes.io/projected/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-api-access-nnnp4\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.732553 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.732523    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls\") pod \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") "
Jul 28 17:55:07.732735 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.732591    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-rbac-proxy-sar-config\") pod \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") "
Jul 28 17:55:07.732799 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.732762    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b54m9\" (UniqueName: \"kubernetes.io/projected/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-api-access-b54m9\") pod \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\" (UID: \"8e664fad-d9a9-4cb8-914e-aa23c663d7aa\") "
Jul 28 17:55:07.732976 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.732953    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8e664fad-d9a9-4cb8-914e-aa23c663d7aa" (UID: "8e664fad-d9a9-4cb8-914e-aa23c663d7aa"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:55:07.733077 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.733058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.733832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.733344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c10a7e1-e62e-4167-b3b4-41d398b449ba-proxy-tls\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.733832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.733397    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnnp4\" (UniqueName: \"kubernetes.io/projected/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-api-access-nnnp4\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.733832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.733493    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:55:07.733832 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.733790    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.734995 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.734967    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8e664fad-d9a9-4cb8-914e-aa23c663d7aa" (UID: "8e664fad-d9a9-4cb8-914e-aa23c663d7aa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:55:07.735753 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.735729    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-api-access-b54m9" (OuterVolumeSpecName: "kube-api-access-b54m9") pod "8e664fad-d9a9-4cb8-914e-aa23c663d7aa" (UID: "8e664fad-d9a9-4cb8-914e-aa23c663d7aa"). InnerVolumeSpecName "kube-api-access-b54m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:55:07.736153 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.736134    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c10a7e1-e62e-4167-b3b4-41d398b449ba-proxy-tls\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.744732 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.744709    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnnp4\" (UniqueName: \"kubernetes.io/projected/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-api-access-nnnp4\") pod \"error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") " pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.835008 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.834971    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:55:07.835008 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.835003    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b54m9\" (UniqueName: \"kubernetes.io/projected/8e664fad-d9a9-4cb8-914e-aa23c663d7aa-kube-api-access-b54m9\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:55:07.869165 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.869130    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:07.905574 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.905542    2556 generic.go:356] "Generic (PLEG): container finished" podID="784170c7-f416-4d44-8c2d-ee625229e312" containerID="eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa" exitCode=2
Jul 28 17:55:07.906048 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.905610    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" event={"ID":"784170c7-f416-4d44-8c2d-ee625229e312","Type":"ContainerDied","Data":"eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa"}
Jul 28 17:55:07.907306 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.907280    2556 generic.go:356] "Generic (PLEG): container finished" podID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerID="656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7" exitCode=0
Jul 28 17:55:07.907422 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.907323    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"
Jul 28 17:55:07.907422 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.907365    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" event={"ID":"8e664fad-d9a9-4cb8-914e-aa23c663d7aa","Type":"ContainerDied","Data":"656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7"}
Jul 28 17:55:07.907422 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.907416    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" event={"ID":"8e664fad-d9a9-4cb8-914e-aa23c663d7aa","Type":"ContainerDied","Data":"c195635c4bdf67cba5b67df75a950f2e8d388dd355fbd3c1a35c93b55ec66b81"}
Jul 28 17:55:07.907594 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.907442    2556 scope.go:117] "RemoveContainer" containerID="6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7"
Jul 28 17:55:07.908402 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.908064    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Jul 28 17:55:07.917974 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.917947    2556 scope.go:117] "RemoveContainer" containerID="656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7"
Jul 28 17:55:07.929983 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.929918    2556 scope.go:117] "RemoveContainer" containerID="6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7"
Jul 28 17:55:07.931508 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:55:07.931474    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7\": container with ID starting with 6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7 not found: ID does not exist" containerID="6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7"
Jul 28 17:55:07.931605 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.931522    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7"} err="failed to get container status \"6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7\": rpc error: code = NotFound desc = could not find container \"6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7\": container with ID starting with 6fd93cea3cdbe0fe65b78c294ae8429b3dba117e7cc52f2773e12d3395f9bcf7 not found: ID does not exist"
Jul 28 17:55:07.931605 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.931553    2556 scope.go:117] "RemoveContainer" containerID="656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7"
Jul 28 17:55:07.931860 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:55:07.931838    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7\": container with ID starting with 656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7 not found: ID does not exist" containerID="656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7"
Jul 28 17:55:07.931936 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.931883    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7"} err="failed to get container status \"656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7\": rpc error: code = NotFound desc = could not find container \"656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7\": container with ID starting with 656ae376788611597335974674055cd5ee77b785be348a9ee65b38270791eeb7 not found: ID does not exist"
Jul 28 17:55:07.934501 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.934470    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"]
Jul 28 17:55:07.939657 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:07.939634    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4"]
Jul 28 17:55:08.025927 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.025902    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"]
Jul 28 17:55:08.028035 ip-10-0-141-165 kubenswrapper[2556]: W0728 17:55:08.028008    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c10a7e1_e62e_4167_b3b4_41d398b449ba.slice/crio-84b3a7021163b8513fc4f178fef4d6ce5698991352fb0937d6c8d82a61eaa531 WatchSource:0}: Error finding container 84b3a7021163b8513fc4f178fef4d6ce5698991352fb0937d6c8d82a61eaa531: Status 404 returned error can't find the container with id 84b3a7021163b8513fc4f178fef4d6ce5698991352fb0937d6c8d82a61eaa531
Jul 28 17:55:08.560771 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.560719    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f7704-predictor-5b75c47dd6-jxdm4" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.45:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 17:55:08.592439 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.592408    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" path="/var/lib/kubelet/pods/8e664fad-d9a9-4cb8-914e-aa23c663d7aa/volumes"
Jul 28 17:55:08.915189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.915093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" event={"ID":"9c10a7e1-e62e-4167-b3b4-41d398b449ba","Type":"ContainerStarted","Data":"92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf"}
Jul 28 17:55:08.915189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.915149    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" event={"ID":"9c10a7e1-e62e-4167-b3b4-41d398b449ba","Type":"ContainerStarted","Data":"4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250"}
Jul 28 17:55:08.915189 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.915171    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" event={"ID":"9c10a7e1-e62e-4167-b3b4-41d398b449ba","Type":"ContainerStarted","Data":"84b3a7021163b8513fc4f178fef4d6ce5698991352fb0937d6c8d82a61eaa531"}
Jul 28 17:55:08.915748 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.915263    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:08.942844 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:08.942786    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podStartSLOduration=1.942769183 podStartE2EDuration="1.942769183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:55:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:55:08.941423539 +0000 UTC m=+1930.887162703" watchObservedRunningTime="2026-07-28 17:55:08.942769183 +0000 UTC m=+1930.888508347"
Jul 28 17:55:09.276586 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:09.276544    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.44:8643/healthz\": dial tcp 10.134.0.44:8643: connect: connection refused"
Jul 28 17:55:09.281977 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:09.281940    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Jul 28 17:55:09.921264 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:09.921229    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:09.922588 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:09.922559    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Jul 28 17:55:10.851701 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.851678    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:55:10.927835 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.927803    2556 generic.go:356] "Generic (PLEG): container finished" podID="784170c7-f416-4d44-8c2d-ee625229e312" containerID="ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f" exitCode=0
Jul 28 17:55:10.928270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.927853    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"
Jul 28 17:55:10.928270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.927901    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" event={"ID":"784170c7-f416-4d44-8c2d-ee625229e312","Type":"ContainerDied","Data":"ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f"}
Jul 28 17:55:10.928270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.927969    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47" event={"ID":"784170c7-f416-4d44-8c2d-ee625229e312","Type":"ContainerDied","Data":"23f2b18dded1f2e8a18723223c970493b620b0b2ff838f11670a5f055d1fdfaa"}
Jul 28 17:55:10.928270 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.927997    2556 scope.go:117] "RemoveContainer" containerID="eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa"
Jul 28 17:55:10.928496 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.928473    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Jul 28 17:55:10.936408 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.936394    2556 scope.go:117] "RemoveContainer" containerID="ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f"
Jul 28 17:55:10.944600 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.944584    2556 scope.go:117] "RemoveContainer" containerID="eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa"
Jul 28 17:55:10.944852 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:55:10.944833    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa\": container with ID starting with eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa not found: ID does not exist" containerID="eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa"
Jul 28 17:55:10.944939 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.944859    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa"} err="failed to get container status \"eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa\": rpc error: code = NotFound desc = could not find container \"eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa\": container with ID starting with eec7ef3f5c2a640fc238d4ba7536dfed4f0b5041c4dbb00181d9d47dc265fcaa not found: ID does not exist"
Jul 28 17:55:10.944939 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.944902    2556 scope.go:117] "RemoveContainer" containerID="ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f"
Jul 28 17:55:10.945145 ip-10-0-141-165 kubenswrapper[2556]: E0728 17:55:10.945126    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f\": container with ID starting with ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f not found: ID does not exist" containerID="ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f"
Jul 28 17:55:10.945194 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.945150    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f"} err="failed to get container status \"ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f\": rpc error: code = NotFound desc = could not find container \"ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f\": container with ID starting with ccde8fd7ff4ad3a64e5160dc9a0c200a1ec8e13c9cdd055dee8c072155eec06f not found: ID does not exist"
Jul 28 17:55:10.962735 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.962709    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8dp4\" (UniqueName: \"kubernetes.io/projected/784170c7-f416-4d44-8c2d-ee625229e312-kube-api-access-b8dp4\") pod \"784170c7-f416-4d44-8c2d-ee625229e312\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") "
Jul 28 17:55:10.962839 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.962753    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/784170c7-f416-4d44-8c2d-ee625229e312-kube-rbac-proxy-sar-config\") pod \"784170c7-f416-4d44-8c2d-ee625229e312\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") "
Jul 28 17:55:10.962922 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.962905    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/784170c7-f416-4d44-8c2d-ee625229e312-proxy-tls\") pod \"784170c7-f416-4d44-8c2d-ee625229e312\" (UID: \"784170c7-f416-4d44-8c2d-ee625229e312\") "
Jul 28 17:55:10.963109 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.963089    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/784170c7-f416-4d44-8c2d-ee625229e312-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "784170c7-f416-4d44-8c2d-ee625229e312" (UID: "784170c7-f416-4d44-8c2d-ee625229e312"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:55:10.963202 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.963189    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/784170c7-f416-4d44-8c2d-ee625229e312-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:55:10.964751 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.964728    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784170c7-f416-4d44-8c2d-ee625229e312-kube-api-access-b8dp4" (OuterVolumeSpecName: "kube-api-access-b8dp4") pod "784170c7-f416-4d44-8c2d-ee625229e312" (UID: "784170c7-f416-4d44-8c2d-ee625229e312"). InnerVolumeSpecName "kube-api-access-b8dp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:55:10.964893 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:10.964850    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/784170c7-f416-4d44-8c2d-ee625229e312-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "784170c7-f416-4d44-8c2d-ee625229e312" (UID: "784170c7-f416-4d44-8c2d-ee625229e312"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:55:11.064158 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:11.064118    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/784170c7-f416-4d44-8c2d-ee625229e312-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:55:11.064158 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:11.064154    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8dp4\" (UniqueName: \"kubernetes.io/projected/784170c7-f416-4d44-8c2d-ee625229e312-kube-api-access-b8dp4\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 17:55:11.256577 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:11.256538    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"]
Jul 28 17:55:11.261723 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:11.261698    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-787a1-predictor-fcb464fdb-zzl47"]
Jul 28 17:55:12.592307 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:12.592272    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784170c7-f416-4d44-8c2d-ee625229e312" path="/var/lib/kubelet/pods/784170c7-f416-4d44-8c2d-ee625229e312/volumes"
Jul 28 17:55:12.912045 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:12.911965    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:12.912447 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:12.912422    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Jul 28 17:55:15.932343 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:15.932315    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:55:15.932760 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:15.932704    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Jul 28 17:55:22.912508 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:22.912467    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Jul 28 17:55:25.933331 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:25.933290    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Jul 28 17:55:32.913290 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:32.913240    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Jul 28 17:55:35.933381 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:35.933342    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Jul 28 17:55:42.912505 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:42.912463    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Jul 28 17:55:45.933334 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:45.933291    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Jul 28 17:55:52.913843 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:52.913769    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 17:55:55.934390 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:55:55.934360    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 17:57:58.616720 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:57:58.616694    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 17:57:58.620820 ip-10-0-141-165 kubenswrapper[2556]: I0728 17:57:58.620797    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:02:58.646907 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:02:58.646863    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:02:58.653497 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:02:58.653471    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:04:18.927642 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:18.927608    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"]
Jul 28 18:04:18.928217 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:18.927836    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container" containerID="cri-o://5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496" gracePeriod=30
Jul 28 18:04:18.928217 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:18.927911    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kube-rbac-proxy" containerID="cri-o://baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510" gracePeriod=30
Jul 28 18:04:19.537833 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:19.537799    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerID="baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510" exitCode=2
Jul 28 18:04:19.538042 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:19.537885    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" event={"ID":"e53e30d3-5717-4f68-b3c4-80fe2ae9be48","Type":"ContainerDied","Data":"baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510"}
Jul 28 18:04:22.187070 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.187045    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 18:04:22.299798 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.299693    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-rbac-proxy-sar-config\") pod \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") "
Jul 28 18:04:22.299798 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.299757    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls\") pod \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") "
Jul 28 18:04:22.300078 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.299910    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqsjg\" (UniqueName: \"kubernetes.io/projected/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-api-access-gqsjg\") pod \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\" (UID: \"e53e30d3-5717-4f68-b3c4-80fe2ae9be48\") "
Jul 28 18:04:22.300196 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.300167    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e53e30d3-5717-4f68-b3c4-80fe2ae9be48" (UID: "e53e30d3-5717-4f68-b3c4-80fe2ae9be48"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:04:22.302012 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.301979    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-api-access-gqsjg" (OuterVolumeSpecName: "kube-api-access-gqsjg") pod "e53e30d3-5717-4f68-b3c4-80fe2ae9be48" (UID: "e53e30d3-5717-4f68-b3c4-80fe2ae9be48"). InnerVolumeSpecName "kube-api-access-gqsjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:04:22.302012 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.301996    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e53e30d3-5717-4f68-b3c4-80fe2ae9be48" (UID: "e53e30d3-5717-4f68-b3c4-80fe2ae9be48"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:04:22.401041 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.401005    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 18:04:22.401041 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.401034    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqsjg\" (UniqueName: \"kubernetes.io/projected/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-api-access-gqsjg\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 18:04:22.401041 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.401044    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e53e30d3-5717-4f68-b3c4-80fe2ae9be48-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 18:04:22.554099 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.553999    2556 generic.go:356] "Generic (PLEG): container finished" podID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerID="5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496" exitCode=0
Jul 28 18:04:22.554099 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.554061    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"
Jul 28 18:04:22.554099 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.554082    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" event={"ID":"e53e30d3-5717-4f68-b3c4-80fe2ae9be48","Type":"ContainerDied","Data":"5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496"}
Jul 28 18:04:22.554331 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.554144    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr" event={"ID":"e53e30d3-5717-4f68-b3c4-80fe2ae9be48","Type":"ContainerDied","Data":"af6944e027f27dc0452576b7125cc499e78e6c06b4af9cb7e11613350f171dbc"}
Jul 28 18:04:22.554331 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.554173    2556 scope.go:117] "RemoveContainer" containerID="baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510"
Jul 28 18:04:22.565718 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.565698    2556 scope.go:117] "RemoveContainer" containerID="5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496"
Jul 28 18:04:22.573659 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.573641    2556 scope.go:117] "RemoveContainer" containerID="baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510"
Jul 28 18:04:22.573929 ip-10-0-141-165 kubenswrapper[2556]: E0728 18:04:22.573911    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510\": container with ID starting with baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510 not found: ID does not exist" containerID="baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510"
Jul 28 18:04:22.573980 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.573939    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510"} err="failed to get container status \"baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510\": rpc error: code = NotFound desc = could not find container \"baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510\": container with ID starting with baf3d5b01d5ab56365b5a4a2bc47c724fc9b65d162f84a03a69f85a3e805e510 not found: ID does not exist"
Jul 28 18:04:22.573980 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.573957    2556 scope.go:117] "RemoveContainer" containerID="5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496"
Jul 28 18:04:22.574226 ip-10-0-141-165 kubenswrapper[2556]: E0728 18:04:22.574207    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496\": container with ID starting with 5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496 not found: ID does not exist" containerID="5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496"
Jul 28 18:04:22.574283 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.574236    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496"} err="failed to get container status \"5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496\": rpc error: code = NotFound desc = could not find container \"5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496\": container with ID starting with 5e673a4b40ee9cc779fc652726ce2fbd39a7a634cd7ae5fb0fa35d256c475496 not found: ID does not exist"
Jul 28 18:04:22.581925 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.581898    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"]
Jul 28 18:04:22.583880 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.583846    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7d999-predictor-566585ff9d-rtwzr"]
Jul 28 18:04:22.590397 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:04:22.590375    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" path="/var/lib/kubelet/pods/e53e30d3-5717-4f68-b3c4-80fe2ae9be48/volumes"
Jul 28 18:07:58.683585 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:07:58.683559    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:07:58.690316 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:07:58.690298    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:12:36.891531 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:36.891453    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"]
Jul 28 18:12:36.892426 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:36.891747    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container" containerID="cri-o://4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250" gracePeriod=30
Jul 28 18:12:36.892426 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:36.892091    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kube-rbac-proxy" containerID="cri-o://92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf" gracePeriod=30
Jul 28 18:12:37.891516 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:37.891482    2556 generic.go:356] "Generic (PLEG): container finished" podID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerID="92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf" exitCode=2
Jul 28 18:12:37.891696 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:37.891543    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" event={"ID":"9c10a7e1-e62e-4167-b3b4-41d398b449ba","Type":"ContainerDied","Data":"92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf"}
Jul 28 18:12:40.035679 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.035652    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 18:12:40.113412 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.113381    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnnp4\" (UniqueName: \"kubernetes.io/projected/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-api-access-nnnp4\") pod \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") "
Jul 28 18:12:40.113590 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.113432    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c10a7e1-e62e-4167-b3b4-41d398b449ba-proxy-tls\") pod \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") "
Jul 28 18:12:40.113590 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.113472    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-rbac-proxy-sar-config\") pod \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\" (UID: \"9c10a7e1-e62e-4167-b3b4-41d398b449ba\") "
Jul 28 18:12:40.113883 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.113843    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9c10a7e1-e62e-4167-b3b4-41d398b449ba" (UID: "9c10a7e1-e62e-4167-b3b4-41d398b449ba"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:12:40.115540 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.115517    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-api-access-nnnp4" (OuterVolumeSpecName: "kube-api-access-nnnp4") pod "9c10a7e1-e62e-4167-b3b4-41d398b449ba" (UID: "9c10a7e1-e62e-4167-b3b4-41d398b449ba"). InnerVolumeSpecName "kube-api-access-nnnp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:12:40.115627 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.115532    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c10a7e1-e62e-4167-b3b4-41d398b449ba-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9c10a7e1-e62e-4167-b3b4-41d398b449ba" (UID: "9c10a7e1-e62e-4167-b3b4-41d398b449ba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:12:40.214560 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.214526    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnnp4\" (UniqueName: \"kubernetes.io/projected/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-api-access-nnnp4\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 18:12:40.214560 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.214559    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c10a7e1-e62e-4167-b3b4-41d398b449ba-proxy-tls\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 18:12:40.214760 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.214570    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c10a7e1-e62e-4167-b3b4-41d398b449ba-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-165.ec2.internal\" DevicePath \"\""
Jul 28 18:12:40.907822 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.907731    2556 generic.go:356] "Generic (PLEG): container finished" podID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerID="4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250" exitCode=0
Jul 28 18:12:40.907822 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.907788    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"
Jul 28 18:12:40.908065 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.907822    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" event={"ID":"9c10a7e1-e62e-4167-b3b4-41d398b449ba","Type":"ContainerDied","Data":"4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250"}
Jul 28 18:12:40.908065 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.907900    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x" event={"ID":"9c10a7e1-e62e-4167-b3b4-41d398b449ba","Type":"ContainerDied","Data":"84b3a7021163b8513fc4f178fef4d6ce5698991352fb0937d6c8d82a61eaa531"}
Jul 28 18:12:40.908065 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.907927    2556 scope.go:117] "RemoveContainer" containerID="92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf"
Jul 28 18:12:40.916486 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.916465    2556 scope.go:117] "RemoveContainer" containerID="4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250"
Jul 28 18:12:40.924352 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.924331    2556 scope.go:117] "RemoveContainer" containerID="92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf"
Jul 28 18:12:40.924613 ip-10-0-141-165 kubenswrapper[2556]: E0728 18:12:40.924593    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf\": container with ID starting with 92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf not found: ID does not exist" containerID="92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf"
Jul 28 18:12:40.924670 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.924624    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf"} err="failed to get container status \"92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf\": rpc error: code = NotFound desc = could not find container \"92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf\": container with ID starting with 92b7e34143f8b6dd02c69129ace2176d409bd8ae3a330c10b7d14b231617c2cf not found: ID does not exist"
Jul 28 18:12:40.924670 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.924647    2556 scope.go:117] "RemoveContainer" containerID="4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250"
Jul 28 18:12:40.924899 ip-10-0-141-165 kubenswrapper[2556]: E0728 18:12:40.924882    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250\": container with ID starting with 4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250 not found: ID does not exist" containerID="4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250"
Jul 28 18:12:40.924947 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.924904    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250"} err="failed to get container status \"4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250\": rpc error: code = NotFound desc = could not find container \"4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250\": container with ID starting with 4edc8fbb4b007d10549cc6c19691eb92b0572fe78c82aeaae813fb038203a250 not found: ID does not exist"
Jul 28 18:12:40.929831 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.929805    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"]
Jul 28 18:12:40.933275 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:40.933253    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cb1ee-predictor-8494ff7d66-sv48x"]
Jul 28 18:12:42.591083 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:42.591052    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" path="/var/lib/kubelet/pods/9c10a7e1-e62e-4167-b3b4-41d398b449ba/volumes"
Jul 28 18:12:58.712345 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:58.712249    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:12:58.729455 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:12:58.719770    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:13:13.026826 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:13.026795    2556 ???:1] "http: TLS handshake error from 10.0.141.165:55848: EOF"
Jul 28 18:13:13.037352 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:13.037311    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t7pdd_a35ecc11-bc89-4ff2-a2e0-53cf022b3ac7/global-pull-secret-syncer/0.log"
Jul 28 18:13:13.151272 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:13.151240    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mt4rt_c544a44d-9a4a-4db5-956f-94711c0a23e3/konnectivity-agent/0.log"
Jul 28 18:13:13.280114 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:13.280030    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-165.ec2.internal_8e2da8fe16979baa8c3430923bef32a9/haproxy/0.log"
Jul 28 18:13:16.949028 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:16.948994    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4e87101-3f3c-40ae-a5d4-894c5f88a0ed/alertmanager/0.log"
Jul 28 18:13:16.996109 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:16.996080    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4e87101-3f3c-40ae-a5d4-894c5f88a0ed/config-reloader/0.log"
Jul 28 18:13:17.042553 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.042517    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4e87101-3f3c-40ae-a5d4-894c5f88a0ed/kube-rbac-proxy-web/0.log"
Jul 28 18:13:17.098139 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.098103    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4e87101-3f3c-40ae-a5d4-894c5f88a0ed/kube-rbac-proxy/0.log"
Jul 28 18:13:17.164213 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.164183    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4e87101-3f3c-40ae-a5d4-894c5f88a0ed/kube-rbac-proxy-metric/0.log"
Jul 28 18:13:17.226238 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.226161    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4e87101-3f3c-40ae-a5d4-894c5f88a0ed/prom-label-proxy/0.log"
Jul 28 18:13:17.282735 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.282703    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4e87101-3f3c-40ae-a5d4-894c5f88a0ed/init-config-reloader/0.log"
Jul 28 18:13:17.363006 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.362977    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-fv7pr_19a545db-ab73-4366-b684-bb8c31012857/cluster-monitoring-operator/0.log"
Jul 28 18:13:17.753446 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.753417    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9lhwz_9149a65e-6937-4fb3-8571-e3079cf03c44/node-exporter/0.log"
Jul 28 18:13:17.780950 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.780926    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9lhwz_9149a65e-6937-4fb3-8571-e3079cf03c44/kube-rbac-proxy/0.log"
Jul 28 18:13:17.833299 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.833272    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9lhwz_9149a65e-6937-4fb3-8571-e3079cf03c44/init-textfile/0.log"
Jul 28 18:13:17.970748 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:17.970703    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6zkkv_fa99f72a-c465-419b-8144-5d91b2742882/kube-rbac-proxy-main/0.log"
Jul 28 18:13:18.008349 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:18.008271    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6zkkv_fa99f72a-c465-419b-8144-5d91b2742882/kube-rbac-proxy-self/0.log"
Jul 28 18:13:18.043688 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:18.043656    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6zkkv_fa99f72a-c465-419b-8144-5d91b2742882/openshift-state-metrics/0.log"
Jul 28 18:13:18.459026 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:18.458998    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-557868565d-5knvj_923d8808-1bde-4536-bce2-f1a6d6229d32/telemeter-client/0.log"
Jul 28 18:13:18.493174 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:18.493141    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-557868565d-5knvj_923d8808-1bde-4536-bce2-f1a6d6229d32/reload/0.log"
Jul 28 18:13:18.529129 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:18.529068    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-557868565d-5knvj_923d8808-1bde-4536-bce2-f1a6d6229d32/kube-rbac-proxy/0.log"
Jul 28 18:13:19.889010 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:19.888979    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-mf42j_11173307-04ab-4ddc-8ed6-473561ddbd80/networking-console-plugin/0.log"
Jul 28 18:13:20.403126 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.403095    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/2.log"
Jul 28 18:13:20.410424 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.410397    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-46tzq_d9260f1e-1598-4199-8917-009e5d328ae6/console-operator/3.log"
Jul 28 18:13:20.862169 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862137    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"]
Jul 28 18:13:20.862525 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862512    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862575 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862529    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862575 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862539    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container"
Jul 28 18:13:20.862575 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862544    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container"
Jul 28 18:13:20.862575 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862554    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container"
Jul 28 18:13:20.862575 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862559    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862566    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862593    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862604    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862610    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862620    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862626    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862638    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862643    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862651    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862657    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862711    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kube-rbac-proxy"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862723    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kserve-container"
Jul 28 18:13:20.862726 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862729    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kube-rbac-proxy"
Jul 28 18:13:20.863180 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862736    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e53e30d3-5717-4f68-b3c4-80fe2ae9be48" containerName="kube-rbac-proxy"
Jul 28 18:13:20.863180 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862745    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="784170c7-f416-4d44-8c2d-ee625229e312" containerName="kserve-container"
Jul 28 18:13:20.863180 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862752    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kserve-container"
Jul 28 18:13:20.863180 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862757    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c10a7e1-e62e-4167-b3b4-41d398b449ba" containerName="kserve-container"
Jul 28 18:13:20.863180 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.862762    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e664fad-d9a9-4cb8-914e-aa23c663d7aa" containerName="kube-rbac-proxy"
Jul 28 18:13:20.865857 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.865841    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.868455 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.868435    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vd4v4\"/\"openshift-service-ca.crt\""
Jul 28 18:13:20.868703 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.868686    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vd4v4\"/\"default-dockercfg-6vjwt\""
Jul 28 18:13:20.869264 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.869246    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vd4v4\"/\"kube-root-ca.crt\""
Jul 28 18:13:20.878536 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.878515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"]
Jul 28 18:13:20.880989 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.880966    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-proc\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.881089 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.881002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-lib-modules\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.881089 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.881080    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-sys\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.881177 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.881129    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmqq4\" (UniqueName: \"kubernetes.io/projected/ddeff1a0-fa45-439b-b7d8-08c833b86dae-kube-api-access-lmqq4\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.881216 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.881194    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-podres\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.890614 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.890594    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-868d45cc84-6cj4k_21a0c0ba-5fb3-4524-87fa-e802174c62a5/console/0.log"
Jul 28 18:13:20.955212 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.955155    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-t97z8_b2e92fd1-5baf-4c86-9a46-ecbc5a19b013/download-server/0.log"
Jul 28 18:13:20.982582 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982546    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-proc\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.982769 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982591    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-lib-modules\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.982769 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982636    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-sys\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.982769 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982685    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmqq4\" (UniqueName: \"kubernetes.io/projected/ddeff1a0-fa45-439b-b7d8-08c833b86dae-kube-api-access-lmqq4\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.982769 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982677    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-proc\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.982769 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-sys\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.982769 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982762    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-podres\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.983080 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982794    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-lib-modules\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.983080 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.982922    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ddeff1a0-fa45-439b-b7d8-08c833b86dae-podres\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:20.991633 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:20.991608    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmqq4\" (UniqueName: \"kubernetes.io/projected/ddeff1a0-fa45-439b-b7d8-08c833b86dae-kube-api-access-lmqq4\") pod \"perf-node-gather-daemonset-4s2lj\" (UID: \"ddeff1a0-fa45-439b-b7d8-08c833b86dae\") " pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:21.175582 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:21.175476    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:21.333251 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:21.333221    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"]
Jul 28 18:13:21.335928 ip-10-0-141-165 kubenswrapper[2556]: W0728 18:13:21.335899    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podddeff1a0_fa45_439b_b7d8_08c833b86dae.slice/crio-e314b8106df69728d817bd7a169dcc5a0ddf9c09b3dc66ef6873a7005c66c745 WatchSource:0}: Error finding container e314b8106df69728d817bd7a169dcc5a0ddf9c09b3dc66ef6873a7005c66c745: Status 404 returned error can't find the container with id e314b8106df69728d817bd7a169dcc5a0ddf9c09b3dc66ef6873a7005c66c745
Jul 28 18:13:21.650103 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:21.650072    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-crtrt_eb9fa04d-1679-4d4e-bef8-77cdb95e07f8/volume-data-source-validator/0.log"
Jul 28 18:13:22.101064 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:22.101029    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj" event={"ID":"ddeff1a0-fa45-439b-b7d8-08c833b86dae","Type":"ContainerStarted","Data":"789db17858db42b71a508eb8857d2abd1bc55ae526a8191d5997029a4b678734"}
Jul 28 18:13:22.101529 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:22.101085    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj" event={"ID":"ddeff1a0-fa45-439b-b7d8-08c833b86dae","Type":"ContainerStarted","Data":"e314b8106df69728d817bd7a169dcc5a0ddf9c09b3dc66ef6873a7005c66c745"}
Jul 28 18:13:22.101529 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:22.101108    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:22.119816 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:22.119762    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj" podStartSLOduration=2.119744515 podStartE2EDuration="2.119744515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:13:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:13:22.118694167 +0000 UTC m=+3024.064433356" watchObservedRunningTime="2026-07-28 18:13:22.119744515 +0000 UTC m=+3024.065483678"
Jul 28 18:13:22.581256 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:22.581224    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b8tjb_b63c8dfa-9627-4721-997b-ae04c040ff9a/dns/0.log"
Jul 28 18:13:22.611031 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:22.610999    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b8tjb_b63c8dfa-9627-4721-997b-ae04c040ff9a/kube-rbac-proxy/0.log"
Jul 28 18:13:22.869187 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:22.869110    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vpgqx_50c96cb9-c926-46eb-8783-993ffbeb9e43/dns-node-resolver/0.log"
Jul 28 18:13:23.339408 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:23.339378    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-9c995f4b7-k4vxw_47ff930e-956f-442b-b72b-bb1fc5a9a9ac/registry/0.log"
Jul 28 18:13:23.427119 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:23.427090    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pq4hj_529a053a-5210-4070-929f-ac5325c9d04b/node-ca/0.log"
Jul 28 18:13:24.607301 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:24.607268    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgv75_017a2f4a-4f71-435a-9d86-3e2e168787fd/serve-healthcheck-canary/0.log"
Jul 28 18:13:24.991372 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:24.991320    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bgkjm_8a3217ae-e76f-4064-bff2-e52f9c649912/insights-operator/0.log"
Jul 28 18:13:24.993560 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:24.993540    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bgkjm_8a3217ae-e76f-4064-bff2-e52f9c649912/insights-operator/1.log"
Jul 28 18:13:25.146851 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:25.146822    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k7br8_c364111e-5375-462e-93bd-cbd691f9b2ef/kube-rbac-proxy/0.log"
Jul 28 18:13:25.171030 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:25.171002    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k7br8_c364111e-5375-462e-93bd-cbd691f9b2ef/exporter/0.log"
Jul 28 18:13:25.198712 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:25.198686    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k7br8_c364111e-5375-462e-93bd-cbd691f9b2ef/extractor/0.log"
Jul 28 18:13:27.703912 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:27.703860    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-cd4d44c67-47s2r_f87467e8-5552-4302-abf1-7c4440cff771/manager/0.log"
Jul 28 18:13:27.763646 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:27.763613    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-d5vl5_917c768f-d5e2-4ceb-93d6-056987b1477f/server/0.log"
Jul 28 18:13:27.908451 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:27.908415    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-75d9cdbddf-922lz_ebd3b847-c5e8-44a7-94eb-48a339ad2e3c/manager/0.log"
Jul 28 18:13:28.010959 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:28.010852    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-tzlm2_5238996d-dbab-41a4-ae10-93b3209e3a09/seaweedfs/0.log"
Jul 28 18:13:28.115673 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:28.115646    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vd4v4/perf-node-gather-daemonset-4s2lj"
Jul 28 18:13:33.104187 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:33.104157    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4_42149cf6-a57a-4cbb-ae05-81d2bd5ac36e/kube-storage-version-migrator-operator/1.log"
Jul 28 18:13:33.105835 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:33.105799    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nwmj4_42149cf6-a57a-4cbb-ae05-81d2bd5ac36e/kube-storage-version-migrator-operator/0.log"
Jul 28 18:13:34.656346 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.656317    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-th8j5_5e969c14-a4ea-49f0-ae8b-db7353c035ce/kube-multus-additional-cni-plugins/0.log"
Jul 28 18:13:34.684922 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.684864    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-th8j5_5e969c14-a4ea-49f0-ae8b-db7353c035ce/egress-router-binary-copy/0.log"
Jul 28 18:13:34.717371 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.717302    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-th8j5_5e969c14-a4ea-49f0-ae8b-db7353c035ce/cni-plugins/0.log"
Jul 28 18:13:34.740990 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.740954    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-th8j5_5e969c14-a4ea-49f0-ae8b-db7353c035ce/bond-cni-plugin/0.log"
Jul 28 18:13:34.767458 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.767424    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-th8j5_5e969c14-a4ea-49f0-ae8b-db7353c035ce/routeoverride-cni/0.log"
Jul 28 18:13:34.792257 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.792226    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-th8j5_5e969c14-a4ea-49f0-ae8b-db7353c035ce/whereabouts-cni-bincopy/0.log"
Jul 28 18:13:34.816327 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.816300    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-th8j5_5e969c14-a4ea-49f0-ae8b-db7353c035ce/whereabouts-cni/0.log"
Jul 28 18:13:34.863781 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:34.863754    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pt279_318d751f-c627-48f7-aa32-2f401c7e5606/kube-multus/0.log"
Jul 28 18:13:35.023598 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:35.023567    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nlpg9_6ae9655b-4180-4a82-9fd7-9eff9b69d437/network-metrics-daemon/0.log"
Jul 28 18:13:35.064223 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:35.064190    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nlpg9_6ae9655b-4180-4a82-9fd7-9eff9b69d437/kube-rbac-proxy/0.log"
Jul 28 18:13:36.459727 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.459695    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/ovn-controller/0.log"
Jul 28 18:13:36.478121 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.478095    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/ovn-acl-logging/0.log"
Jul 28 18:13:36.506635 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.506604    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/kube-rbac-proxy-node/0.log"
Jul 28 18:13:36.536512 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.536477    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/kube-rbac-proxy-ovn-metrics/0.log"
Jul 28 18:13:36.558432 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.558404    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/northd/0.log"
Jul 28 18:13:36.584289 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.584261    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/nbdb/0.log"
Jul 28 18:13:36.608586 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.608561    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/sbdb/0.log"
Jul 28 18:13:36.675137 ip-10-0-141-165 kubenswrapper[2556]: I0728 18:13:36.675100    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrdnt_7f20edee-328b-4193-87de-90ad2efc3bb7/ovnkube-controller/0.log"