Sep 17 15:39:13.921397 ip-10-0-140-183 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 15:39:14.270227 ip-10-0-140-183 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 15:39:14.270227 ip-10-0-140-183 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 15:39:14.270227 ip-10-0-140-183 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 15:39:14.270227 ip-10-0-140-183 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 15:39:14.270227 ip-10-0-140-183 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 15:39:14.271186 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.271084    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 15:39:14.275291 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275274    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:39:14.275291 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275289    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:39:14.275291 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275294    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275298    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275301    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275304    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275307    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275309    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275312    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275315    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275318    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275326    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275329    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275332    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275335    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275337    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275340    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275343    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275345    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275348    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275350    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:39:14.275387 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275353    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275356    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275359    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275361    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275364    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275366    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275369    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275372    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275374    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275376    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275379    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275382    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275385    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275389    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275393    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275396    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275400    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275404    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275407    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275409    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:39:14.275837 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275412    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275415    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275418    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275420    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275423    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275425    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275428    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275430    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275433    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275436    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275438    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275442    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275444    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275447    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275451    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275454    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275458    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275462    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275465    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275468    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:39:14.276345 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275471    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275474    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275477    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275480    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275483    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275486    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275489    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275492    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275494    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275497    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275500    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275502    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275505    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275508    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275512    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275514    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275517    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275519    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275523    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:39:14.276830 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275526    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275528    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275531    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275533    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275536    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275539    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275541    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275544    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275547    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275550    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275552    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275555    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275558    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275561    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275563    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275566    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275569    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275571    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.275574    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276265    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:39:14.277313 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276272    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276275    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276277    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276280    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276283    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276286    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276289    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276292    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276294    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276297    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276300    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276303    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276305    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276310    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276314    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276317    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276320    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276324    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276328    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:39:14.277803 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276331    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276335    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276338    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276341    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276344    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276347    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276350    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276353    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276355    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276358    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276361    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276364    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276366    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276368    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276371    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276374    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276376    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276379    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276382    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276385    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:39:14.278281 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276388    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276391    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276393    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276396    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276399    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276401    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276404    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276406    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276409    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276411    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276414    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276416    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276419    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276422    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276425    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276427    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276429    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276432    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276435    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276437    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:39:14.278768 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276439    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276442    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276444    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276447    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276450    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276453    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276455    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276457    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276460    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276464    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276466    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276469    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276471    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276474    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276477    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276479    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276482    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276484    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276487    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:39:14.279265 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276489    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276509    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276513    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276517    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276521    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276524    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276527    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276530    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276533    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276536    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276541    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276543    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276546    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276549    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276551    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276554    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276556    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276559    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276562    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:39:14.279750 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276564    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276909    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276915    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276919    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276922    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276926    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276928    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276931    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276935    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276939    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276942    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276945    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276948    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276951    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276954    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276957    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276960    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276963    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276965    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:39:14.280221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276968    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276971    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276973    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276976    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276979    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276983    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276985    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276988    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276990    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276993    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276996    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.276998    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277001    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277004    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277006    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277009    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277011    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277013    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277016    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277019    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:39:14.280692 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277022    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277025    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277027    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277030    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277033    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277035    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277038    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277040    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277043    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277046    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277049    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277051    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277054    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277057    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277059    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277062    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277065    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277067    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277070    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277072    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:39:14.281215 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277075    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277078    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277080    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277083    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277085    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277088    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277104    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277107    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277110    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277114    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277118    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277121    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277124    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277127    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277129    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277132    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277134    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277137    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277139    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277142    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:39:14.281734 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277145    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277148    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277151    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277154    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277156    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277159    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277162    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277165    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277167    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277170    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277173    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277175    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277178    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277181    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277183    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277186    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277188    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277191    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277193    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277196    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:39:14.282250 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277199    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277261    2568 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277293    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277300    2568 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277304    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277310    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277313    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277317    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277322    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277325    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277328    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277331    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277334    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277338    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277341    2568 flags.go:64] FLAG: --cgroup-root=""
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277344    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277347    2568 flags.go:64] FLAG: --client-ca-file=""
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277350    2568 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277353    2568 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277357    2568 flags.go:64] FLAG: --cluster-domain=""
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277360    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277363    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277366    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277369    2568 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 15:39:14.282762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277373    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277376    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277379    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277383    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277386    2568 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277389    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277392    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277395    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277398    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277401    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277405    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277407    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277410    2568 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277413    2568 flags.go:64] FLAG: --enable-server="true"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277417    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277421    2568 flags.go:64] FLAG: --event-burst="100"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277425    2568 flags.go:64] FLAG: --event-qps="50"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277427    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277430    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277433    2568 flags.go:64] FLAG: --eviction-hard=""
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277439    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277442    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277445    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277448    2568 flags.go:64] FLAG: --eviction-soft=""
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277452    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 15:39:14.283352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277455    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277458    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277461    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277464    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277466    2568 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277469    2568 flags.go:64] FLAG: --feature-gates=""
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277473    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277476    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277479    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277482    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277485    2568 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277488    2568 flags.go:64] FLAG: --help="false"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277491    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277494    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277497    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277500    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277503    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277507    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277509    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277514    2568 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277517    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277520    2568 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277524    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277527    2568 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 15:39:14.283985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277530    2568 flags.go:64] FLAG: --kube-reserved=""
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277533    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277536    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277540    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277543    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277546    2568 flags.go:64] FLAG: --lock-file=""
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277548    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277551    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277554    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277560    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277563    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277566    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277568    2568 flags.go:64] FLAG: --logging-format="text"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277571    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277575    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277578    2568 flags.go:64] FLAG: --manifest-url=""
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277581    2568 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277585    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277588    2568 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277592    2568 flags.go:64] FLAG: --max-pods="110"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277595    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277598    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277601    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277604    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277607    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 15:39:14.284589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277610    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277613    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277620    2568 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277623    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277626    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277630    2568 flags.go:64] FLAG: --pod-cidr=""
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277634    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277639    2568 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277642    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277645    2568 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277650    2568 flags.go:64] FLAG: --port="10250"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277653    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277656    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-015d62dfaccee789f"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277659    2568 flags.go:64] FLAG: --qos-reserved=""
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277662    2568 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277665    2568 flags.go:64] FLAG: --register-node="true"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277668    2568 flags.go:64] FLAG: --register-with-taints=""
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277671    2568 flags.go:64] FLAG: --registry-burst="10"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277674    2568 flags.go:64] FLAG: --registry-qps="5"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277677    2568 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277680    2568 flags.go:64] FLAG: --reserved-memory=""
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277683    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277686    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277689    2568 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277692    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 15:39:14.285204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277695    2568 flags.go:64] FLAG: --runonce="false"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277698    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277700    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277703    2568 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277706    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277709    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277712    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277715    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277718    2568 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277721    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277726    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277729    2568 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277732    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277735    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277738    2568 flags.go:64] FLAG: --system-cgroups=""
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277750    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277756    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277760    2568 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277763    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277767    2568 flags.go:64] FLAG: --tls-min-version=""
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277770    2568 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277773    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277776    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277779    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277782    2568 flags.go:64] FLAG: --v="2"
Sep 17 15:39:14.285843 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277786    2568 flags.go:64] FLAG: --version="false"
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277790    2568 flags.go:64] FLAG: --vmodule=""
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277793    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.277797    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277906    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277910    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277913    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277916    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277919    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277921    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277924    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277927    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277930    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277932    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277936    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277940    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277943    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277947    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277951    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277953    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:39:14.286461 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277956    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277959    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277962    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277965    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277969    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277972    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277974    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277977    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277980    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277990    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277993    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277995    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.277999    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278003    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278006    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278009    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278012    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278015    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278018    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:39:14.286948 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278021    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278024    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278027    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278029    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278032    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278035    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278037    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278040    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278042    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278045    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278049    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278051    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278054    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278057    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278059    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278062    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278065    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278069    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278072    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278075    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:39:14.287446 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278077    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278080    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278082    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278085    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278087    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278109    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278112    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278114    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278117    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278119    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278122    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278125    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278127    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278130    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278133    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278135    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278138    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278141    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278143    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278146    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:39:14.287930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278148    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278151    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278155    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278158    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278160    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278164    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278166    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278169    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278172    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278176    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278179    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278182    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278184    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278187    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278190    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278193    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278197    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278200    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278202    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278205    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:39:14.288512 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278207    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278210    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278213    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.278215    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.278220    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.284836    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.284852    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.285793    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.285806    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285877    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285882    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285885    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285888    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285891    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285894    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:39:14.288997 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285897    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285900    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285903    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285906    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285908    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285911    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285914    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285917    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285920    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285922    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285925    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285928    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285930    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285933    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285935    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285938    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285940    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285943    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285946    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285949    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285952    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:39:14.289427 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285955    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285958    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285961    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285964    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285966    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285969    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285972    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285975    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285977    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285980    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285982    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285985    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285988    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285990    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285993    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285995    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.285998    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286001    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286003    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286006    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:39:14.289952 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286008    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286011    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286013    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286016    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286019    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286021    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286024    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286026    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286029    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286031    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286034    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286037    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286041    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286043    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286046    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286049    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286052    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286054    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286057    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:39:14.290457 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286061    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286065    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286068    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286071    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286073    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286076    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286079    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286082    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286084    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286088    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286102    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286106    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286110    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286114    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286118    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286121    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286124    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286127    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286130    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:39:14.290918 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286145    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286149    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286152    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286155    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286159    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286162    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286165    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286168    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286171    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286174    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286178    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286181    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286184    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286186    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:39:14.291381 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.286192    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286303    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286307    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286310    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286313    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286316    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286319    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286322    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286325    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286328    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286331    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286333    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286336    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286338    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286341    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286344    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286346    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286349    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286352    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286354    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:39:14.291720 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286357    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286360    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286362    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286365    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286368    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286370    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286373    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286375    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286378    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286382    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286384    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286387    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286389    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286392    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286394    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286397    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286399    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286402    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286404    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286407    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:39:14.292221 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286409    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286412    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286414    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286417    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286419    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286422    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286425    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286427    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286430    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286432    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286435    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286437    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286440    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286442    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286445    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286447    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286451    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286454    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286457    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286460    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:39:14.292760 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286462    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286465    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286468    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286471    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286474    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286477    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286479    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286482    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286484    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286487    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286489    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286492    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286495    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286497    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286500    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286502    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286505    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286507    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286511    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286515    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:39:14.293360 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286518    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286520    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286523    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286525    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286528    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286531    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286533    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286536    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286538    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286541    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286544    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286546    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286549    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286552    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286554    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286557    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286559    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286562    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286566    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:39:14.293995 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:14.286569    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:39:14.294480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.286574    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 15:39:14.294480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.286673    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 15:39:14.294480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.288623    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 15:39:14.294480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.289526    2568 server.go:1019] "Starting client certificate rotation"
Sep 17 15:39:14.294480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.289629    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 15:39:14.294480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.289661    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 15:39:14.294480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.291856    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 15:39:14.297156 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.297129    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 15:39:14.307078 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.307064    2568 log.go:25] "Validated CRI v1 runtime API"
Sep 17 15:39:14.311815 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.311801    2568 log.go:25] "Validated CRI v1 image API"
Sep 17 15:39:14.312680 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.312664    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 15:39:14.315372 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.315348    2568 fs.go:135] Filesystem UUIDs: map[04f37d39-1616-4c16-8be8-28e084c18245:/dev/nvme0n1p3 798add5b-6ba0-4842-a54f-ec679c3a1df4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 17 15:39:14.315442 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.315371    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 15:39:14.320373 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.320266    2568 manager.go:216] Machine: {Timestamp:2026-09-17 15:39:14.318964576 +0000 UTC m=+0.294753369 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099865 MemoryCapacity:33164468224 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22650beddd27a67130a2df5bde61d5 SystemUUID:ec22650b-eddd-27a6-7130-a2df5bde61d5 BootID:bbacf652-8826-4369-8ce1-f023495b2c0e Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f7:b0:eb:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f7:b0:eb:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:f0:92:5c:d2:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164468224 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 15:39:14.320373 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.320365    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 15:39:14.320536 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.320463    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 15:39:14.322984 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.322957    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 15:39:14.323190 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.322987    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-183.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 15:39:14.323783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.323771    2568 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 15:39:14.323842 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.323791    2568 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 15:39:14.323842 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.323802    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 15:39:14.323842 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.323822    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 15:39:14.323842 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.323832    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 15:39:14.324748 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.324736    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 15:39:14.327526 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.327512    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 15:39:14.327653 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.327643    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 15:39:14.327708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.327678    2568 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 15:39:14.327708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.327698    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 15:39:14.327790 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.327718    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 15:39:14.327790 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.327735    2568 kubelet.go:398] "Adding apiserver pod source"
Sep 17 15:39:14.327790 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.327747    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 15:39:14.328115 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.328082    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 15:39:14.328207 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.328123    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 15:39:14.330193 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.330173    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 17 15:39:14.333516 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.333495    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 15:39:14.333578 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.333536    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 15:39:14.334957 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.334943    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 15:39:14.334995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.334967    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 15:39:14.334995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.334976    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 15:39:14.334995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.334981    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 15:39:14.334995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.334990    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 15:39:14.334995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.334996    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 15:39:14.335145 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335010    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 15:39:14.335145 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335015    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 15:39:14.335145 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335023    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 15:39:14.335145 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335029    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 15:39:14.335145 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335040    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 15:39:14.335145 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335045    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 15:39:14.335893 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335882    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 15:39:14.335893 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.335892    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 15:39:14.338663 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.338643    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-183.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 15:39:14.338733 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.338667    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 15:39:14.338733 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.338714    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-183.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 15:39:14.339573 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.339562    2568 server.go:1268] "Started kubelet"
Sep 17 15:39:14.339758 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.339650    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 15:39:14.339758 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.339652    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 15:39:14.339758 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.339714    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 15:39:14.340504 ip-10-0-140-183 systemd[1]: Started Kubernetes Kubelet.
Sep 17 15:39:14.341701 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.341263    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 15:39:14.341701 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.341475    2568 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 15:39:14.344225 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.344204    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 15:39:14.345690 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.344740    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-183.ec2.internal.18d625e1f54ac409  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-183.ec2.internal,UID:ip-10-0-140-183.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-183.ec2.internal,},FirstTimestamp:2026-09-17 15:39:14.339537929 +0000 UTC m=+0.315326723,LastTimestamp:2026-09-17 15:39:14.339537929 +0000 UTC m=+0.315326723,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-183.ec2.internal,}"
Sep 17 15:39:14.346508 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.346490    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 15:39:14.347214 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.347197    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 15:39:14.347337 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.347314    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:14.347890 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.347867    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 15:39:14.347890 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.347889    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 15:39:14.347890 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.347869    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 15:39:14.348056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.347987    2568 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 15:39:14.348056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.347996    2568 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 15:39:14.348553 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.348458    2568 factory.go:55] Registering systemd factory
Sep 17 15:39:14.348553 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.348473    2568 factory.go:223] Registration of the systemd container factory successfully
Sep 17 15:39:14.348729 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.348710    2568 factory.go:153] Registering CRI-O factory
Sep 17 15:39:14.348729 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.348729    2568 factory.go:223] Registration of the crio container factory successfully
Sep 17 15:39:14.348880 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.348779    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 15:39:14.348880 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.348797    2568 factory.go:103] Registering Raw factory
Sep 17 15:39:14.348880 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.348814    2568 manager.go:1196] Started watching for new ooms in manager
Sep 17 15:39:14.349289 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.349274    2568 manager.go:349] Starting recovery of all containers
Sep 17 15:39:14.349580 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.349539    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jnvx2"
Sep 17 15:39:14.351240 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.351214    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-183.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 15:39:14.351240 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.351227    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 15:39:14.359335 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.359316    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jnvx2"
Sep 17 15:39:14.363150 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.363133    2568 manager.go:354] Recovery completed
Sep 17 15:39:14.369052 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.368921    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:14.371878 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.371850    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:14.371956 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.371899    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:14.371956 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.371913    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:14.372445 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.372431    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 15:39:14.372445 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.372444    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 15:39:14.372554 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.372451    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:14.372554 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.372460    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 15:39:14.373116 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.373088    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:14.373185 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.373127    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:14.373185 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.373137    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:14.373719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.373708    2568 policy_none.go:49] "None policy: Start"
Sep 17 15:39:14.373758 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.373725    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 15:39:14.373758 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.373737    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 15:39:14.374522 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.374465    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-183.ec2.internal.18d625e1f73850da  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-183.ec2.internal,UID:ip-10-0-140-183.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-183.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-183.ec2.internal,},FirstTimestamp:2026-09-17 15:39:14.371883226 +0000 UTC m=+0.347672020,LastTimestamp:2026-09-17 15:39:14.371883226 +0000 UTC m=+0.347672020,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-183.ec2.internal,}"
Sep 17 15:39:14.375011 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.374998    2568 policy_none.go:47] "Start"
Sep 17 15:39:14.398698 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.398676    2568 manager.go:337] "Starting Device Plugin manager"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.398710    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.398720    2568 server.go:87] "Starting device plugin registration server"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.399088    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.399128    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.399228    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.399297    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.399308    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.399928    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 15:39:14.412848 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.399982    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:14.435811 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.435788    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 15:39:14.436957 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.436940    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 15:39:14.437021 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.436997    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 15:39:14.437021 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.437020    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 15:39:14.437165 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.437132    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 15:39:14.439399 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.439367    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:39:14.499477 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.499457    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:14.500302 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.500286    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:14.500386 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.500317    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:14.500386 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.500331    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:14.500386 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.500349    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.509419 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.509406    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.509464 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.509428    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-183.ec2.internal\": node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:14.528031 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.527985    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:14.537566 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.537533    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"]
Sep 17 15:39:14.537665 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.537606    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:14.538362 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.538340    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:14.538415 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.538369    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:14.538415 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.538379    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:14.539576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.539564    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:14.539714 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.539698    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.539765 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.539728    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:14.540244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.540228    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:14.540315 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.540252    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:14.540315 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.540265    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:14.540315 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.540307    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:14.540420 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.540329    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:14.540420 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.540344    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:14.541373 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.541359    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.541434 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.541382    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:14.542483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.542471    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:14.542552 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.542493    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:14.542552 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.542506    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:14.549065 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.549050    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78a56846a26782d79f6f1b96091c0b40-config\") pod \"kube-apiserver-proxy-ip-10-0-140-183.ec2.internal\" (UID: \"78a56846a26782d79f6f1b96091c0b40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.549160 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.549074    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2948c7237ca7a7061402e7afd2e99d8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"2948c7237ca7a7061402e7afd2e99d8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.549160 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.549109    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2948c7237ca7a7061402e7afd2e99d8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"2948c7237ca7a7061402e7afd2e99d8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.563615 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.563605    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-183.ec2.internal\" not found" node="ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.567817 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.567801    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-183.ec2.internal\" not found" node="ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.628218 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.628192    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:14.649388 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.649364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78a56846a26782d79f6f1b96091c0b40-config\") pod \"kube-apiserver-proxy-ip-10-0-140-183.ec2.internal\" (UID: \"78a56846a26782d79f6f1b96091c0b40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.649464 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.649393    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2948c7237ca7a7061402e7afd2e99d8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"2948c7237ca7a7061402e7afd2e99d8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.649464 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.649411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2948c7237ca7a7061402e7afd2e99d8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"2948c7237ca7a7061402e7afd2e99d8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.649464 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.649457    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78a56846a26782d79f6f1b96091c0b40-config\") pod \"kube-apiserver-proxy-ip-10-0-140-183.ec2.internal\" (UID: \"78a56846a26782d79f6f1b96091c0b40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.649558 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.649470    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2948c7237ca7a7061402e7afd2e99d8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"2948c7237ca7a7061402e7afd2e99d8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.649558 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.649506    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2948c7237ca7a7061402e7afd2e99d8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"2948c7237ca7a7061402e7afd2e99d8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.728912 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.728878    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:14.829609 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.829541    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:14.866155 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.866118    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.870163 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:14.870126    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:14.930254 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:14.930217    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:15.030709 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:15.030680    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:15.131312 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:15.131227    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:16.619470 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:16.619435    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:16.668767 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.668742    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:39:16.677101 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.677079    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 15:39:16.677236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.677215    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 15:39:16.677286 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.677238    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 15:39:16.720569 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:16.720535    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:16.742125 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:16.742093    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a56846a26782d79f6f1b96091c0b40.slice/crio-87c9318361d11bb651a0459819ec191b48dc20c3e5497fc8c694897a0bb5de68 WatchSource:0}: Error finding container 87c9318361d11bb651a0459819ec191b48dc20c3e5497fc8c694897a0bb5de68: Status 404 returned error can't find the container with id 87c9318361d11bb651a0459819ec191b48dc20c3e5497fc8c694897a0bb5de68
Sep 17 15:39:16.743849 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:16.743809    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2948c7237ca7a7061402e7afd2e99d8e.slice/crio-0972fd8ef0508279d9507c35b21609b8c807f20f31c5fdc49d9db6d2660c24d9 WatchSource:0}: Error finding container 0972fd8ef0508279d9507c35b21609b8c807f20f31c5fdc49d9db6d2660c24d9: Status 404 returned error can't find the container with id 0972fd8ef0508279d9507c35b21609b8c807f20f31c5fdc49d9db6d2660c24d9
Sep 17 15:39:16.748573 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.748556    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 15:39:16.751298 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.751273    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 15:34:15 +0000 UTC" deadline="2028-07-02 15:30:16.216039836 +0000 UTC"
Sep 17 15:39:16.751361 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.751298    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15695h50m59.464745394s"
Sep 17 15:39:16.821248 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:16.821171    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:16.829597 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.829556    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" event={"ID":"2948c7237ca7a7061402e7afd2e99d8e","Type":"ContainerStarted","Data":"0972fd8ef0508279d9507c35b21609b8c807f20f31c5fdc49d9db6d2660c24d9"}
Sep 17 15:39:16.830578 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.830554    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal" event={"ID":"78a56846a26782d79f6f1b96091c0b40","Type":"ContainerStarted","Data":"87c9318361d11bb651a0459819ec191b48dc20c3e5497fc8c694897a0bb5de68"}
Sep 17 15:39:16.922102 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:16.922070    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:16.923148 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.923135    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:39:16.924169 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.924150    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:39:16.924260 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.924180    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:39:16.924260 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:16.924192    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:39:17.023149 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.023117    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:17.107792 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.107713    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:39:17.123719 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.123691    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Sep 17 15:39:17.185067 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.185034    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:39:17.235780 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.235750    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:17.262218 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.262188    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 15:39:17.263148 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.263123    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Sep 17 15:39:17.281583 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.281552    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 15:39:17.506170 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.506087    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:39:17.716755 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.716719    2568 apiserver.go:52] "Watching apiserver"
Sep 17 15:39:17.725752 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.725724    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 15:39:17.727575 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.727544    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-c5x5h","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv","openshift-cluster-node-tuning-operator/tuned-7m8pw","openshift-dns/node-resolver-2k5tc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal","openshift-multus/multus-htrfs","openshift-multus/network-metrics-daemon-cgmvm","openshift-network-operator/iptables-alerter-ldwnn","kube-system/global-pull-secret-syncer-9btfr","kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal","openshift-image-registry/node-ca-kf4b5","openshift-multus/multus-additional-cni-plugins-ql6ch","openshift-network-diagnostics/network-check-target-b5xqc","openshift-ovn-kubernetes/ovnkube-node-fm9p8"]
Sep 17 15:39:17.728994 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.728971    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:17.729078 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.729055    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:17.730107 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.730080    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.731306 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.731223    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.732269 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.732247    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.732900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.732879    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 15:39:17.733002 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.732898    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 15:39:17.733977 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.733648    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 15:39:17.733977 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.733659    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5pl2q\""
Sep 17 15:39:17.733977 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.733798    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:39:17.734179 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.733984    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.735028 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.735006    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 15:39:17.736799 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.736151    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 15:39:17.736799 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.736264    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.736799 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.736652    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lkc9v\""
Sep 17 15:39:17.738257 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.738238    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.738884 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.738747    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 15:39:17.739281 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739116    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 15:39:17.739281 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739146    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 15:39:17.739281 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739162    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9gw5b\""
Sep 17 15:39:17.739281 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739170    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 15:39:17.739281 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739175    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 15:39:17.739543 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739294    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 15:39:17.739543 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739535    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 15:39:17.739604 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.739586    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:17.739670 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.739647    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:17.740499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.740138    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:39:17.740499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.740148    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qwmpb\""
Sep 17 15:39:17.740499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.740162    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 15:39:17.740499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.740457    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mmv47\""
Sep 17 15:39:17.740672 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.740527    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 15:39:17.740672 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.740645    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 15:39:17.741156 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.741138    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:17.742759 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.742742    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.743616 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.743595    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 15:39:17.744415 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.744398    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.744506 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.744458    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:17.745999 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.745960    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 15:39:17.745999 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.745987    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.746585 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746558    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk9sx\" (UniqueName: \"kubernetes.io/projected/bd02d50f-d630-4da8-877c-dbb32eba205c-kube-api-access-zk9sx\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.746675 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746588    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.746675 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746614    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-host\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.746675 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746637    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eda22602-f2c3-4cab-8e46-4835749a5b97-hosts-file\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.746788 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746683    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 15:39:17.746788 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746683    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:17.746788 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746730    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysctl-d\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.746788 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746753    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqwsw\" (UniqueName: \"kubernetes.io/projected/2c529498-cf03-4d4a-8e37-a23f28cbee5a-kube-api-access-rqwsw\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.746788 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.747015 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746802    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.747015 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-serviceca\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.747015 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746882    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-modprobe-d\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.747015 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746906    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-node-log\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.747015 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746928    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-cni-netd\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.747015 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746952    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbbcp\" (UniqueName: \"kubernetes.io/projected/7f90604f-1cf6-45ea-aa28-fd27b066f800-kube-api-access-hbbcp\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.747015 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.746990    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-system-cni-dir\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747032    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747038    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747035    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-cnibin\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747094    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda22602-f2c3-4cab-8e46-4835749a5b97-tmp-dir\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747113    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747125    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdxnq\" (UniqueName: \"kubernetes.io/projected/eda22602-f2c3-4cab-8e46-4835749a5b97-kube-api-access-rdxnq\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747146    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-sys\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747183    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qvsdv\""
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747195    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-var-lib-kubelet\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747216    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-host\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747220    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747250    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-kubelet\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747274    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-cni-bin\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747308    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysconfig\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747035    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747331    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-tuned\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.747359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747351    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-slash\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747360    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q8wkw\""
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747423    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd02d50f-d630-4da8-877c-dbb32eba205c-ovn-node-metrics-cert\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747447    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb5l9\" (UniqueName: \"kubernetes.io/projected/b8aadb40-1dda-4451-8765-ca03203e05a0-kube-api-access-bb5l9\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747482    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5m8x\" (UniqueName: \"kubernetes.io/projected/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-kube-api-access-r5m8x\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747528    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgqtv\" (UniqueName: \"kubernetes.io/projected/7ff164ba-34fa-45aa-b854-e48384d80808-kube-api-access-cgqtv\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747554    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-run\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-env-overrides\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747600    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-ovnkube-script-lib\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747626    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f90604f-1cf6-45ea-aa28-fd27b066f800-host-slash\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747649    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747695    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysctl-conf\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747718    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-lib-modules\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747740    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-systemd-units\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748238 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747763    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-run-netns\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747784    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-ovn\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747863    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-log-socket\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747926    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-run-ovn-kubernetes\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747952    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.747954    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-kubernetes\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748056    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-systemd\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748085    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7f90604f-1cf6-45ea-aa28-fd27b066f800-iptables-alerter-script\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748109    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-systemd\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748149    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c529498-cf03-4d4a-8e37-a23f28cbee5a-tmp\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748180    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-var-lib-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748188    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jwjxk\""
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748204    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-etc-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748227    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-ovnkube-config\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748272    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4fplc\""
Sep 17 15:39:17.748900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.748600    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 15:39:17.750165 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.750144    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 15:39:17.752051 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.752031    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 15:39:17.772108 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.772081    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mqjp4"
Sep 17 15:39:17.780480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.780464    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mqjp4"
Sep 17 15:39:17.836341 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.836312    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 15:39:17.848904 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.848877    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk9sx\" (UniqueName: \"kubernetes.io/projected/bd02d50f-d630-4da8-877c-dbb32eba205c-kube-api-access-zk9sx\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.849022 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.848919    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-cni-multus\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.849022 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.848946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:17.849022 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.848967    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysctl-d\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.849022 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.848990    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p4vt\" (UniqueName: \"kubernetes.io/projected/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-kube-api-access-2p4vt\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.849022 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849013    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68386cc8-defe-4273-8d2e-2746d342d9d3-cni-binary-copy\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849033    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-socket-dir-parent\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849057    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.849058    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849107    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-node-log\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849118    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysctl-d\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.849171    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:18.349131036 +0000 UTC m=+2.937175613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849171    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-node-log\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849200    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbbcp\" (UniqueName: \"kubernetes.io/projected/7f90604f-1cf6-45ea-aa28-fd27b066f800-kube-api-access-hbbcp\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849231    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-system-cni-dir\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849254    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-device-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.849270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849271    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-cni-bin\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849283    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-conf-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849308    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-system-cni-dir\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849338    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdxnq\" (UniqueName: \"kubernetes.io/projected/eda22602-f2c3-4cab-8e46-4835749a5b97-kube-api-access-rdxnq\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-sys\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849391    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-host\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849419    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-sys\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-kubelet\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849457    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-host\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849462    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-socket-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849461    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-kubelet\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849501    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-tuned\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849530    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb5l9\" (UniqueName: \"kubernetes.io/projected/b8aadb40-1dda-4451-8765-ca03203e05a0-kube-api-access-bb5l9\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849559    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-sys-fs\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849586    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-system-cni-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849611    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-ovnkube-script-lib\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.849787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849642    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/437c724d-ac60-4db7-861a-704df11b37c9-dbus\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849666    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-cnibin\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849689    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-multus-certs\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849708    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysctl-conf\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849724    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-lib-modules\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849745    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-log-socket\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849763    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-run-ovn-kubernetes\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849786    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b69d31ab-249a-4aa8-b4bb-4245e9183d62-agent-certs\") pod \"konnectivity-agent-c5x5h\" (UID: \"b69d31ab-249a-4aa8-b4bb-4245e9183d62\") " pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849818    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-log-socket\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849889    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysctl-conf\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849949    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-lib-modules\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849947    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-run-ovn-kubernetes\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.849981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850005    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7f90604f-1cf6-45ea-aa28-fd27b066f800-iptables-alerter-script\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850030    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-k8s-cni-cncf-io\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850057    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-systemd\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.850667 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850082    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-etc-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850115    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-ovnkube-config\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850134    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-etc-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850170    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850262    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-ovnkube-script-lib\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850295    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-systemd\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850339    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-host\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850372    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-host\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850403    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eda22602-f2c3-4cab-8e46-4835749a5b97-hosts-file\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850432    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqwsw\" (UniqueName: \"kubernetes.io/projected/2c529498-cf03-4d4a-8e37-a23f28cbee5a-kube-api-access-rqwsw\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850469    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850478    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eda22602-f2c3-4cab-8e46-4835749a5b97-hosts-file\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850522    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b69d31ab-249a-4aa8-b4bb-4245e9183d62-konnectivity-ca\") pod \"konnectivity-agent-c5x5h\" (UID: \"b69d31ab-249a-4aa8-b4bb-4245e9183d62\") " pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850543    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850545    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-etc-kubernetes\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.851576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850585    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850615    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-serviceca\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850624    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-ovnkube-config\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850640    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-modprobe-d\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850694    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-cni-netd\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850725    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-cnibin\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850749    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-cni-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850766    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-modprobe-d\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t58wq\" (UniqueName: \"kubernetes.io/projected/68386cc8-defe-4273-8d2e-2746d342d9d3-kube-api-access-t58wq\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850777    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850809    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda22602-f2c3-4cab-8e46-4835749a5b97-tmp-dir\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850825    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8aadb40-1dda-4451-8765-ca03203e05a0-cnibin\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850833    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-var-lib-kubelet\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-cni-netd\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850877    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-cni-bin\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850902    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850909    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-var-lib-kubelet\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.852219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850930    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-hostroot\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850949    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-cni-bin\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850956    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-daemon-config\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysconfig\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.850987    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851004    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-slash\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd02d50f-d630-4da8-877c-dbb32eba205c-ovn-node-metrics-cert\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851049    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-sysconfig\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851052    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-serviceca\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851054    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-registration-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851088    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-slash\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5m8x\" (UniqueName: \"kubernetes.io/projected/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-kube-api-access-r5m8x\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851106    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851131    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgqtv\" (UniqueName: \"kubernetes.io/projected/7ff164ba-34fa-45aa-b854-e48384d80808-kube-api-access-cgqtv\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851132    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b8aadb40-1dda-4451-8765-ca03203e05a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851131    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda22602-f2c3-4cab-8e46-4835749a5b97-tmp-dir\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851154    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-run\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.852936 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851219    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-env-overrides\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851194    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-run\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851245    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f90604f-1cf6-45ea-aa28-fd27b066f800-host-slash\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851249    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7f90604f-1cf6-45ea-aa28-fd27b066f800-iptables-alerter-script\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851282    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f90604f-1cf6-45ea-aa28-fd27b066f800-host-slash\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851311    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851342    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-systemd-units\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-run-netns\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851380    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-systemd-units\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-ovn\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851419    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/437c724d-ac60-4db7-861a-704df11b37c9-kubelet-config\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851426    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-host-run-netns\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851446    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-kubelet\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851470    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-kubernetes\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851490    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-ovn\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851491    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-systemd\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851525    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-run-systemd\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.853622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851537    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-etc-selinux\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851554    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd02d50f-d630-4da8-877c-dbb32eba205c-env-overrides\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851564    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-netns\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851586    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-kubernetes\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c529498-cf03-4d4a-8e37-a23f28cbee5a-tmp\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851632    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-var-lib-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.851711    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd02d50f-d630-4da8-877c-dbb32eba205c-var-lib-openvswitch\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.853820    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c529498-cf03-4d4a-8e37-a23f28cbee5a-etc-tuned\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.853863    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c529498-cf03-4d4a-8e37-a23f28cbee5a-tmp\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.854329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.853975    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd02d50f-d630-4da8-877c-dbb32eba205c-ovn-node-metrics-cert\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.907520 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.907489    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbbcp\" (UniqueName: \"kubernetes.io/projected/7f90604f-1cf6-45ea-aa28-fd27b066f800-kube-api-access-hbbcp\") pod \"iptables-alerter-ldwnn\" (UID: \"7f90604f-1cf6-45ea-aa28-fd27b066f800\") " pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:17.908856 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.908814    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqwsw\" (UniqueName: \"kubernetes.io/projected/2c529498-cf03-4d4a-8e37-a23f28cbee5a-kube-api-access-rqwsw\") pod \"tuned-7m8pw\" (UID: \"2c529498-cf03-4d4a-8e37-a23f28cbee5a\") " pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:17.909261 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.909239    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5m8x\" (UniqueName: \"kubernetes.io/projected/1896bca3-c7f4-40bd-99c1-4a3f6d1a4365-kube-api-access-r5m8x\") pod \"node-ca-kf4b5\" (UID: \"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365\") " pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:17.919194 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.919166    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb5l9\" (UniqueName: \"kubernetes.io/projected/b8aadb40-1dda-4451-8765-ca03203e05a0-kube-api-access-bb5l9\") pod \"multus-additional-cni-plugins-ql6ch\" (UID: \"b8aadb40-1dda-4451-8765-ca03203e05a0\") " pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:17.923140 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.923108    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdxnq\" (UniqueName: \"kubernetes.io/projected/eda22602-f2c3-4cab-8e46-4835749a5b97-kube-api-access-rdxnq\") pod \"node-resolver-2k5tc\" (UID: \"eda22602-f2c3-4cab-8e46-4835749a5b97\") " pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:17.925212 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.925191    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgqtv\" (UniqueName: \"kubernetes.io/projected/7ff164ba-34fa-45aa-b854-e48384d80808-kube-api-access-cgqtv\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:17.941070 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.941047    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk9sx\" (UniqueName: \"kubernetes.io/projected/bd02d50f-d630-4da8-877c-dbb32eba205c-kube-api-access-zk9sx\") pod \"ovnkube-node-fm9p8\" (UID: \"bd02d50f-d630-4da8-877c-dbb32eba205c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:17.952287 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952263    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-conf-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952423 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952301    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-socket-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.952423 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952325    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-sys-fs\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.952423 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952372    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-system-cni-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952423 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952385    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-conf-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952423 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952408    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/437c724d-ac60-4db7-861a-704df11b37c9-dbus\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952436    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-cnibin\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-multus-certs\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952462    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-system-cni-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952464    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-sys-fs\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952464    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/437c724d-ac60-4db7-861a-704df11b37c9-dbus\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952475    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b69d31ab-249a-4aa8-b4bb-4245e9183d62-agent-certs\") pod \"konnectivity-agent-c5x5h\" (UID: \"b69d31ab-249a-4aa8-b4bb-4245e9183d62\") " pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952506    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-cnibin\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952518    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952542    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-k8s-cni-cncf-io\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952553    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-multus-certs\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952563    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952572    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-socket-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952590    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b69d31ab-249a-4aa8-b4bb-4245e9183d62-konnectivity-ca\") pod \"konnectivity-agent-c5x5h\" (UID: \"b69d31ab-249a-4aa8-b4bb-4245e9183d62\") " pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952608    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-etc-kubernetes\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952638    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-cni-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952624    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-k8s-cni-cncf-io\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.952668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952662    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t58wq\" (UniqueName: \"kubernetes.io/projected/68386cc8-defe-4273-8d2e-2746d342d9d3-kube-api-access-t58wq\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952707    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-etc-kubernetes\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.952726    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952738    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-cni-dir\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952773    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-hostroot\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.952792    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret podName:437c724d-ac60-4db7-861a-704df11b37c9 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:18.452773616 +0000 UTC m=+3.040818199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret") pod "global-pull-secret-syncer-9btfr" (UID: "437c724d-ac60-4db7-861a-704df11b37c9") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952820    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-daemon-config\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952828    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-hostroot\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-registration-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952919    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/437c724d-ac60-4db7-861a-704df11b37c9-kubelet-config\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952972    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-kubelet\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.952985    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-registration-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953002    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-etc-selinux\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953027    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-netns\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953042    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/437c724d-ac60-4db7-861a-704df11b37c9-kubelet-config\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953054    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-cni-multus\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.953475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953078    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953093    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p4vt\" (UniqueName: \"kubernetes.io/projected/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-kube-api-access-2p4vt\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68386cc8-defe-4273-8d2e-2746d342d9d3-cni-binary-copy\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953125    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-etc-selinux\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953134    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-run-netns\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953141    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-socket-dir-parent\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953156    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-kubelet\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953196    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-device-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953209    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-cni-multus\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-cni-bin\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953290    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-host-var-lib-cni-bin\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-socket-dir-parent\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953366    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-device-dir\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953369    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68386cc8-defe-4273-8d2e-2746d342d9d3-multus-daemon-config\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953678    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68386cc8-defe-4273-8d2e-2746d342d9d3-cni-binary-copy\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:17.954104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.953876    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b69d31ab-249a-4aa8-b4bb-4245e9183d62-konnectivity-ca\") pod \"konnectivity-agent-c5x5h\" (UID: \"b69d31ab-249a-4aa8-b4bb-4245e9183d62\") " pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:17.954991 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.954974    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b69d31ab-249a-4aa8-b4bb-4245e9183d62-agent-certs\") pod \"konnectivity-agent-c5x5h\" (UID: \"b69d31ab-249a-4aa8-b4bb-4245e9183d62\") " pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:17.977277 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.977254    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:39:17.977377 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.977283    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:39:17.977377 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.977298    2568 projected.go:196] Error preparing data for projected volume kube-api-access-rgvbq for pod openshift-network-diagnostics/network-check-target-b5xqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:17.977377 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:17.977361    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq podName:a8098f77-1464-43c8-b25f-4991d2c44f90 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:18.477348653 +0000 UTC m=+3.065393217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rgvbq" (UniqueName: "kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq") pod "network-check-target-b5xqc" (UID: "a8098f77-1464-43c8-b25f-4991d2c44f90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:17.980278 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.980253    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p4vt\" (UniqueName: \"kubernetes.io/projected/dcd56591-9f49-4061-a37e-ab8e8fdb0a14-kube-api-access-2p4vt\") pod \"aws-ebs-csi-driver-node-prckv\" (UID: \"dcd56591-9f49-4061-a37e-ab8e8fdb0a14\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:17.982390 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:17.982370    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t58wq\" (UniqueName: \"kubernetes.io/projected/68386cc8-defe-4273-8d2e-2746d342d9d3-kube-api-access-t58wq\") pod \"multus-htrfs\" (UID: \"68386cc8-defe-4273-8d2e-2746d342d9d3\") " pod="openshift-multus/multus-htrfs"
Sep 17 15:39:18.042988 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.042910    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kf4b5"
Sep 17 15:39:18.051275 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.051250    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7m8pw"
Sep 17 15:39:18.061935 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.061903    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2k5tc"
Sep 17 15:39:18.066403 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.066381    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:18.073833 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.073818    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ldwnn"
Sep 17 15:39:18.080330 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.080312    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ql6ch"
Sep 17 15:39:18.087597 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.087579    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:18.095217 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.095200    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv"
Sep 17 15:39:18.100730 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.100712    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-htrfs"
Sep 17 15:39:18.296417 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.296342    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:39:18.356196 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.356161    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:18.356371 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.356314    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:18.356438 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.356382    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:19.356364064 +0000 UTC m=+3.944408633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:18.457036 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.457008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:18.457191 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.457125    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:18.457191 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.457181    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret podName:437c724d-ac60-4db7-861a-704df11b37c9 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:19.457164295 +0000 UTC m=+4.045208860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret") pod "global-pull-secret-syncer-9btfr" (UID: "437c724d-ac60-4db7-861a-704df11b37c9") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:18.557825 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.557748    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:18.557981 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.557881    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:39:18.557981 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.557898    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:39:18.557981 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.557906    2568 projected.go:196] Error preparing data for projected volume kube-api-access-rgvbq for pod openshift-network-diagnostics/network-check-target-b5xqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:18.557981 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:18.557955    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq podName:a8098f77-1464-43c8-b25f-4991d2c44f90 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:19.557942122 +0000 UTC m=+4.145986686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgvbq" (UniqueName: "kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq") pod "network-check-target-b5xqc" (UID: "a8098f77-1464-43c8-b25f-4991d2c44f90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:18.585523 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.585490    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8aadb40_1dda_4451_8765_ca03203e05a0.slice/crio-21dbea2c7169d90050b17d457a33fc70693005551ce99c27b082c06523d88e3b WatchSource:0}: Error finding container 21dbea2c7169d90050b17d457a33fc70693005551ce99c27b082c06523d88e3b: Status 404 returned error can't find the container with id 21dbea2c7169d90050b17d457a33fc70693005551ce99c27b082c06523d88e3b
Sep 17 15:39:18.586989 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.586901    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd02d50f_d630_4da8_877c_dbb32eba205c.slice/crio-2ae6ae6f52c4134b03fd1fa611bee6fa36e5c2aad6b233d2faa15f2a3d6fd84c WatchSource:0}: Error finding container 2ae6ae6f52c4134b03fd1fa611bee6fa36e5c2aad6b233d2faa15f2a3d6fd84c: Status 404 returned error can't find the container with id 2ae6ae6f52c4134b03fd1fa611bee6fa36e5c2aad6b233d2faa15f2a3d6fd84c
Sep 17 15:39:18.590391 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.590246    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f90604f_1cf6_45ea_aa28_fd27b066f800.slice/crio-e118a48770f4a248411929fec2fcc8676e27108b04897bc442bf1a5b7f429a5b WatchSource:0}: Error finding container e118a48770f4a248411929fec2fcc8676e27108b04897bc442bf1a5b7f429a5b: Status 404 returned error can't find the container with id e118a48770f4a248411929fec2fcc8676e27108b04897bc442bf1a5b7f429a5b
Sep 17 15:39:18.591543 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.591512    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1896bca3_c7f4_40bd_99c1_4a3f6d1a4365.slice/crio-6d5998c81dec39a7c13244e7f81c91a2fcf56d2a38eca33a77ea97464452941e WatchSource:0}: Error finding container 6d5998c81dec39a7c13244e7f81c91a2fcf56d2a38eca33a77ea97464452941e: Status 404 returned error can't find the container with id 6d5998c81dec39a7c13244e7f81c91a2fcf56d2a38eca33a77ea97464452941e
Sep 17 15:39:18.612128 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.612105    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c529498_cf03_4d4a_8e37_a23f28cbee5a.slice/crio-ed2cd0f02147623aa1938433e90f06e4cb34ddb4ebfdb8dc813be9c6011543b9 WatchSource:0}: Error finding container ed2cd0f02147623aa1938433e90f06e4cb34ddb4ebfdb8dc813be9c6011543b9: Status 404 returned error can't find the container with id ed2cd0f02147623aa1938433e90f06e4cb34ddb4ebfdb8dc813be9c6011543b9
Sep 17 15:39:18.612940 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.612885    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd56591_9f49_4061_a37e_ab8e8fdb0a14.slice/crio-dc23dd3e729a66e29e9ac55adcbd746d22d0d7da44fce32d2ce5d73d257a263c WatchSource:0}: Error finding container dc23dd3e729a66e29e9ac55adcbd746d22d0d7da44fce32d2ce5d73d257a263c: Status 404 returned error can't find the container with id dc23dd3e729a66e29e9ac55adcbd746d22d0d7da44fce32d2ce5d73d257a263c
Sep 17 15:39:18.614690 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.614668    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69d31ab_249a_4aa8_b4bb_4245e9183d62.slice/crio-d044f6d152141d985d8a2e1aabc558e7bae906334af3995c6160addf1af4c718 WatchSource:0}: Error finding container d044f6d152141d985d8a2e1aabc558e7bae906334af3995c6160addf1af4c718: Status 404 returned error can't find the container with id d044f6d152141d985d8a2e1aabc558e7bae906334af3995c6160addf1af4c718
Sep 17 15:39:18.614981 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.614958    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda22602_f2c3_4cab_8e46_4835749a5b97.slice/crio-0eb4a2b48bcd485f6b4b843d256835f7c595bdff0bc771bcb89e0e4595b93bca WatchSource:0}: Error finding container 0eb4a2b48bcd485f6b4b843d256835f7c595bdff0bc771bcb89e0e4595b93bca: Status 404 returned error can't find the container with id 0eb4a2b48bcd485f6b4b843d256835f7c595bdff0bc771bcb89e0e4595b93bca
Sep 17 15:39:18.615882 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:18.615863    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68386cc8_defe_4273_8d2e_2746d342d9d3.slice/crio-4eecee4c451813e8fa64ba217fd7ce436ff9554aa53773f4ca334753090e5e65 WatchSource:0}: Error finding container 4eecee4c451813e8fa64ba217fd7ce436ff9554aa53773f4ca334753090e5e65: Status 404 returned error can't find the container with id 4eecee4c451813e8fa64ba217fd7ce436ff9554aa53773f4ca334753090e5e65
Sep 17 15:39:18.783068 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.783015    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 15:34:17 +0000 UTC" deadline="2028-03-11 03:25:45.065783558 +0000 UTC"
Sep 17 15:39:18.783068 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.783063    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12971h46m26.282723221s"
Sep 17 15:39:18.837000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.836971    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5x5h" event={"ID":"b69d31ab-249a-4aa8-b4bb-4245e9183d62","Type":"ContainerStarted","Data":"d044f6d152141d985d8a2e1aabc558e7bae906334af3995c6160addf1af4c718"}
Sep 17 15:39:18.837974 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.837954    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv" event={"ID":"dcd56591-9f49-4061-a37e-ab8e8fdb0a14","Type":"ContainerStarted","Data":"dc23dd3e729a66e29e9ac55adcbd746d22d0d7da44fce32d2ce5d73d257a263c"}
Sep 17 15:39:18.838901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.838882    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7m8pw" event={"ID":"2c529498-cf03-4d4a-8e37-a23f28cbee5a","Type":"ContainerStarted","Data":"ed2cd0f02147623aa1938433e90f06e4cb34ddb4ebfdb8dc813be9c6011543b9"}
Sep 17 15:39:18.839766 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.839748    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ldwnn" event={"ID":"7f90604f-1cf6-45ea-aa28-fd27b066f800","Type":"ContainerStarted","Data":"e118a48770f4a248411929fec2fcc8676e27108b04897bc442bf1a5b7f429a5b"}
Sep 17 15:39:18.840737 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.840718    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"2ae6ae6f52c4134b03fd1fa611bee6fa36e5c2aad6b233d2faa15f2a3d6fd84c"}
Sep 17 15:39:18.841797 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.841777    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerStarted","Data":"21dbea2c7169d90050b17d457a33fc70693005551ce99c27b082c06523d88e3b"}
Sep 17 15:39:18.843290 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.843262    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal" event={"ID":"78a56846a26782d79f6f1b96091c0b40","Type":"ContainerStarted","Data":"5e112acb6ed940c1df3bf35088a16402cb0c279f6ecc7d1819d1ecd3d739f91e"}
Sep 17 15:39:18.844592 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.844569    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-htrfs" event={"ID":"68386cc8-defe-4273-8d2e-2746d342d9d3","Type":"ContainerStarted","Data":"4eecee4c451813e8fa64ba217fd7ce436ff9554aa53773f4ca334753090e5e65"}
Sep 17 15:39:18.848822 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.848780    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2k5tc" event={"ID":"eda22602-f2c3-4cab-8e46-4835749a5b97","Type":"ContainerStarted","Data":"0eb4a2b48bcd485f6b4b843d256835f7c595bdff0bc771bcb89e0e4595b93bca"}
Sep 17 15:39:18.849962 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.849939    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kf4b5" event={"ID":"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365","Type":"ContainerStarted","Data":"6d5998c81dec39a7c13244e7f81c91a2fcf56d2a38eca33a77ea97464452941e"}
Sep 17 15:39:18.875440 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:18.875384    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal" podStartSLOduration=1.875367202 podStartE2EDuration="1.875367202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:18.874939211 +0000 UTC m=+3.462983797" watchObservedRunningTime="2026-09-17 15:39:18.875367202 +0000 UTC m=+3.463411790"
Sep 17 15:39:19.364661 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.364621    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:19.364862 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.364806    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:19.364938 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.364881    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:21.364863707 +0000 UTC m=+5.952908276 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:19.466150 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.466112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:19.466317 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.466261    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:19.466389 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.466322    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret podName:437c724d-ac60-4db7-861a-704df11b37c9 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:21.466302793 +0000 UTC m=+6.054347365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret") pod "global-pull-secret-syncer-9btfr" (UID: "437c724d-ac60-4db7-861a-704df11b37c9") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:19.567118 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.566468    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:19.567118 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.566675    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:39:19.567118 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.566696    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:39:19.567118 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.566709    2568 projected.go:196] Error preparing data for projected volume kube-api-access-rgvbq for pod openshift-network-diagnostics/network-check-target-b5xqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:19.567118 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.566762    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq podName:a8098f77-1464-43c8-b25f-4991d2c44f90 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:21.56674626 +0000 UTC m=+6.154790827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgvbq" (UniqueName: "kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq") pod "network-check-target-b5xqc" (UID: "a8098f77-1464-43c8-b25f-4991d2c44f90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:19.784113 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.783978    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 15:34:17 +0000 UTC" deadline="2028-06-09 10:09:46.652593408 +0000 UTC"
Sep 17 15:39:19.784113 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.784037    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15138h30m26.868560377s"
Sep 17 15:39:19.831738 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.831712    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:19.831982 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.831905    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:19.832339 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.832317    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:19.832455 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.832412    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:19.832508 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.832487    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:19.832612 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:19.832562    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:19.857146 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.857118    2568 generic.go:356] "Generic (PLEG): container finished" podID="2948c7237ca7a7061402e7afd2e99d8e" containerID="129f0cf98978d3b607a14ae40d971273ef21140c6be2a26bcccdc8dddc35b77a" exitCode=0
Sep 17 15:39:19.858169 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:19.858136    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" event={"ID":"2948c7237ca7a7061402e7afd2e99d8e","Type":"ContainerDied","Data":"129f0cf98978d3b607a14ae40d971273ef21140c6be2a26bcccdc8dddc35b77a"}
Sep 17 15:39:20.873862 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:20.873603    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" event={"ID":"2948c7237ca7a7061402e7afd2e99d8e","Type":"ContainerStarted","Data":"136eb3cdfac3bbe924811ca6dc53fd162686d1942c329de4a325467f1c6a360e"}
Sep 17 15:39:21.381664 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:21.381626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:21.381881 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.381782    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:21.381881 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.381864    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:25.381829197 +0000 UTC m=+9.969873773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:21.483106 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:21.483066    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:21.483305 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.483231    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:21.483305 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.483296    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret podName:437c724d-ac60-4db7-861a-704df11b37c9 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:25.483277536 +0000 UTC m=+10.071322106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret") pod "global-pull-secret-syncer-9btfr" (UID: "437c724d-ac60-4db7-861a-704df11b37c9") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:21.584449 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:21.584408    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:21.584709 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.584688    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:39:21.584816 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.584718    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:39:21.584816 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.584732    2568 projected.go:196] Error preparing data for projected volume kube-api-access-rgvbq for pod openshift-network-diagnostics/network-check-target-b5xqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:21.584816 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.584795    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq podName:a8098f77-1464-43c8-b25f-4991d2c44f90 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:25.584775953 +0000 UTC m=+10.172820525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgvbq" (UniqueName: "kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq") pod "network-check-target-b5xqc" (UID: "a8098f77-1464-43c8-b25f-4991d2c44f90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:21.825510 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:21.825423    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:21.825510 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:21.825466    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:21.825724 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:21.825427    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:21.825724 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.825579    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:21.825724 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.825700    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:21.825898 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:21.825782    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:23.825727 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:23.825689    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:23.826197 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:23.825827    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:23.826279 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:23.826243    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:23.826356 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:23.826334    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:23.826484 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:23.826470    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:23.826572 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:23.826554    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:25.418502 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:25.418464    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:25.418976 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.418670    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:25.418976 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.418736    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:33.418718612 +0000 UTC m=+18.006763202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:25.519717 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:25.519683    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:25.519895 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.519792    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:25.519895 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.519881    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret podName:437c724d-ac60-4db7-861a-704df11b37c9 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:33.519826912 +0000 UTC m=+18.107871476 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret") pod "global-pull-secret-syncer-9btfr" (UID: "437c724d-ac60-4db7-861a-704df11b37c9") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:25.620228 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:25.620188    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:25.620400 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.620378    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:39:25.620475 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.620405    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:39:25.620475 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.620418    2568 projected.go:196] Error preparing data for projected volume kube-api-access-rgvbq for pod openshift-network-diagnostics/network-check-target-b5xqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:25.620576 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.620481    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq podName:a8098f77-1464-43c8-b25f-4991d2c44f90 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:33.620461445 +0000 UTC m=+18.208506025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgvbq" (UniqueName: "kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq") pod "network-check-target-b5xqc" (UID: "a8098f77-1464-43c8-b25f-4991d2c44f90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:25.826804 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:25.826002    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:25.826804 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.826113    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:25.826804 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:25.826460    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:25.826804 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.826552    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:25.826804 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:25.826596    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:25.826804 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:25.826682    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:27.825851 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:27.825800    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:27.826298 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:27.825800    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:27.826298 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:27.825962    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:27.826298 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:27.825812    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:27.826298 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:27.826032    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:27.826298 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:27.826106    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:29.825051 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:29.825020    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:29.825051 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:29.825042    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:29.825573 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:29.825121    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:29.825573 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:29.825137    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:29.825573 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:29.825214    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:29.825573 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:29.825286    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:31.825730 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:31.825684    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:31.826193 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:31.825684    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:31.826193 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:31.825823    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:31.826193 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:31.825691    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:31.826193 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:31.825921    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:31.826193 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:31.826052    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:33.474729 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:33.474699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:33.475170 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.474827    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:33.475170 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.474900    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:49.474884198 +0000 UTC m=+34.062928762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:33.575630 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:33.575589    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:33.575816 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.575740    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:33.575816 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.575809    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret podName:437c724d-ac60-4db7-861a-704df11b37c9 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:49.57579155 +0000 UTC m=+34.163836128 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret") pod "global-pull-secret-syncer-9btfr" (UID: "437c724d-ac60-4db7-861a-704df11b37c9") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:33.676577 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:33.676538    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:33.676759 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.676740    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:39:33.676805 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.676764    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:39:33.676805 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.676774    2568 projected.go:196] Error preparing data for projected volume kube-api-access-rgvbq for pod openshift-network-diagnostics/network-check-target-b5xqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:33.676890 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.676821    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq podName:a8098f77-1464-43c8-b25f-4991d2c44f90 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:49.676807931 +0000 UTC m=+34.264852495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgvbq" (UniqueName: "kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq") pod "network-check-target-b5xqc" (UID: "a8098f77-1464-43c8-b25f-4991d2c44f90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:33.825259 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:33.825172    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:33.825259 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:33.825217    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:33.825459 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:33.825178    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:33.825459 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.825299    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:33.825459 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.825375    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:33.825612 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:33.825466    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:35.825908 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:35.825878    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:35.826353 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:35.825915    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:35.826353 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:35.825976    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:35.826353 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:35.826042    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:35.826353 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:35.826090    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:35.826353 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:35.826180    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:36.907995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.907966    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-htrfs" event={"ID":"68386cc8-defe-4273-8d2e-2746d342d9d3","Type":"ContainerStarted","Data":"dfb3607450e645f5c4657047b3f95936d4203f746307e13bf1e98882656c1115"}
Sep 17 15:39:36.909503 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.909479    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2k5tc" event={"ID":"eda22602-f2c3-4cab-8e46-4835749a5b97","Type":"ContainerStarted","Data":"a4309fb1ba667a8cb419de4ea18df8c532dbb59eea367a4075aec80d4c4cd7ec"}
Sep 17 15:39:36.910974 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.910953    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kf4b5" event={"ID":"1896bca3-c7f4-40bd-99c1-4a3f6d1a4365","Type":"ContainerStarted","Data":"19a17705e62c13f8a2ff5e6da8a5763c4fc0054cee4d0fec26f372f70b40affb"}
Sep 17 15:39:36.912327 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.912309    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5x5h" event={"ID":"b69d31ab-249a-4aa8-b4bb-4245e9183d62","Type":"ContainerStarted","Data":"ef35aaed18fba2d368ced1b6d1f38ce30fc46d14be4a5644bd1c4e7a6e21532f"}
Sep 17 15:39:36.913813 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.913797    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv" event={"ID":"dcd56591-9f49-4061-a37e-ab8e8fdb0a14","Type":"ContainerStarted","Data":"13912df7bde1f31936f9c1891e3daefb8edb119912b5325dc4e91f0400d2668c"}
Sep 17 15:39:36.915117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.915099    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7m8pw" event={"ID":"2c529498-cf03-4d4a-8e37-a23f28cbee5a","Type":"ContainerStarted","Data":"d9fc6758dc28a9b41d012cac3d63b24c39bbbc9a26dd954cbbb94f144f7f11eb"}
Sep 17 15:39:36.918460 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.918430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"ea1dca4d1ed1a2a0512727cf48a449664e05ea1851de4d15c54cfc936399c16c"}
Sep 17 15:39:36.918588 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.918506    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"49a0d2acf5b263bc5503dccba7f4cec5f6265b90f2d75bc7364b2a59c547b0f3"}
Sep 17 15:39:36.918588 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.918557    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"aad21598a5728d7d10076620ad01a1e9133db6488f62f704244ce845476e08ef"}
Sep 17 15:39:36.918815 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.918604    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"f8727065cc67487d10d3fb7f225a9ca9f88daa9c1fdb282bc2c1c853fcdd0f26"}
Sep 17 15:39:36.920206 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.920187    2568 generic.go:356] "Generic (PLEG): container finished" podID="b8aadb40-1dda-4451-8765-ca03203e05a0" containerID="99e635a712beada656ef3dcffbfb8dbc0334dff90cdffa78c7bb6efc8302f688" exitCode=0
Sep 17 15:39:36.920283 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.920214    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerDied","Data":"99e635a712beada656ef3dcffbfb8dbc0334dff90cdffa78c7bb6efc8302f688"}
Sep 17 15:39:36.935896 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.935829    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" podStartSLOduration=19.935813233 podStartE2EDuration="19.935813233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:20.896549266 +0000 UTC m=+5.484593857" watchObservedRunningTime="2026-09-17 15:39:36.935813233 +0000 UTC m=+21.523857823"
Sep 17 15:39:36.935985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.935940    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-htrfs" podStartSLOduration=2.9594853949999997 podStartE2EDuration="20.935933068s" totalImagesPullingTime="17.976447673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:36.935778849 +0000 UTC m=+21.523823438" watchObservedRunningTime="2026-09-17 15:39:36.935933068 +0000 UTC m=+21.523977656"
Sep 17 15:39:36.980124 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.980090    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7m8pw" podStartSLOduration=4.213491735 podStartE2EDuration="21.980076558s" totalImagesPullingTime="17.766584823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:36.962938828 +0000 UTC m=+21.550983416" watchObservedRunningTime="2026-09-17 15:39:36.980076558 +0000 UTC m=+21.568121145"
Sep 17 15:39:36.980206 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.980173    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2k5tc" podStartSLOduration=4.197844634 podStartE2EDuration="21.980168255s" totalImagesPullingTime="17.782323621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:36.979791579 +0000 UTC m=+21.567836166" watchObservedRunningTime="2026-09-17 15:39:36.980168255 +0000 UTC m=+21.568212843"
Sep 17 15:39:36.997669 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:36.997633    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kf4b5" podStartSLOduration=4.210555186 podStartE2EDuration="21.997620156s" totalImagesPullingTime="17.78706497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:36.997535167 +0000 UTC m=+21.585579756" watchObservedRunningTime="2026-09-17 15:39:36.997620156 +0000 UTC m=+21.585664786"
Sep 17 15:39:37.753865 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.753337    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 15:39:37.805704 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.805536    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T15:39:37.753357301Z","UUID":"75aa0a98-1524-40a3-bbc1-9df9d0c79bf0","Handler":null,"Name":"","Endpoint":""}
Sep 17 15:39:37.807535 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.807514    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 15:39:37.807631 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.807549    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 15:39:37.825657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.825633    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:37.825744 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.825657    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:37.825744 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.825633    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:37.825831 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:37.825798    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:37.825976 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:37.825955    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:37.826064 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:37.826047    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:37.924328 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.924294    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv" event={"ID":"dcd56591-9f49-4061-a37e-ab8e8fdb0a14","Type":"ContainerStarted","Data":"1303178ae4098fcf7e2c018436ca5b76acde853929eb0aabaa4dbeac7130605c"}
Sep 17 15:39:37.925969 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.925941    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ldwnn" event={"ID":"7f90604f-1cf6-45ea-aa28-fd27b066f800","Type":"ContainerStarted","Data":"144b18eac7be485d7cb619845d7bce9f6bf91e41e34bfa86568f60a840e0dec0"}
Sep 17 15:39:37.932190 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.932157    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"f2221dd3fb6c4dbc84995bd7a30ee3e81b1c3b8eef24adbcb0e1b60771be28fa"}
Sep 17 15:39:37.932308 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.932288    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"6e14c362500432483df286818393b2f86bca656284074c5651c04beb189f1a86"}
Sep 17 15:39:37.952761 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:37.952685    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c5x5h" podStartSLOduration=5.22076086 podStartE2EDuration="22.952671377s" totalImagesPullingTime="17.731910517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:37.078484186 +0000 UTC m=+21.666528772" watchObservedRunningTime="2026-09-17 15:39:37.952671377 +0000 UTC m=+22.540715963"
Sep 17 15:39:39.826118 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:39.826083    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:39.826118 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:39.826104    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:39.826681 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:39.826210    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:39.826681 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:39.826280    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:39.826681 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:39.826408    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:39.826681 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:39.826518    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:39.939166 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:39.939123    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv" event={"ID":"dcd56591-9f49-4061-a37e-ab8e8fdb0a14","Type":"ContainerStarted","Data":"03d8a3a1ca1b892aebc291582d76ac735984ec38c77d383cb6dc4bd85e04c963"}
Sep 17 15:39:39.942976 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:39.942944    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"567bf22aa1bb3258c89c1a01b0946b0ebe959d20cd8edd89e0d1792877ca9d4a"}
Sep 17 15:39:39.963205 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:39.963159    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ldwnn" podStartSLOduration=7.176758065 podStartE2EDuration="24.96314158s" totalImagesPullingTime="17.786383515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:37.953202954 +0000 UTC m=+22.541247537" watchObservedRunningTime="2026-09-17 15:39:39.96314158 +0000 UTC m=+24.551186169"
Sep 17 15:39:39.963626 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:39.963596    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prckv" podStartSLOduration=4.003028248 podStartE2EDuration="23.963585067s" totalImagesPullingTime="19.960556819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:39.962944499 +0000 UTC m=+24.550989085" watchObservedRunningTime="2026-09-17 15:39:39.963585067 +0000 UTC m=+24.551629654"
Sep 17 15:39:40.834240 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:40.834206    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:40.834927 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:40.834905    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:41.291337 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.291075    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:41.292248 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.292224    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c5x5h"
Sep 17 15:39:41.825557 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.825369    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:41.825719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.825369    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:41.825719 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:41.825648    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:41.825719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.825428    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:41.825908 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:41.825697    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:41.825908 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:41.825768    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:41.949691 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.949657    2568 generic.go:356] "Generic (PLEG): container finished" podID="b8aadb40-1dda-4451-8765-ca03203e05a0" containerID="daf2293c2dcef46b1e785dd719d65475dd9d73c9e14ee78de04627b44edbe185" exitCode=0
Sep 17 15:39:41.950139 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.949736    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerDied","Data":"daf2293c2dcef46b1e785dd719d65475dd9d73c9e14ee78de04627b44edbe185"}
Sep 17 15:39:41.955406 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:41.955366    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" event={"ID":"bd02d50f-d630-4da8-877c-dbb32eba205c","Type":"ContainerStarted","Data":"b316134deeb71b4c8898b0fe34879cafc2e3f7be314aa16e3d6097f65b66c513"}
Sep 17 15:39:42.010617 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.010573    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8" podStartSLOduration=9.195969755 podStartE2EDuration="27.010558133s" totalImagesPullingTime="17.814588378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:42.010094692 +0000 UTC m=+26.598139271" watchObservedRunningTime="2026-09-17 15:39:42.010558133 +0000 UTC m=+26.598602717"
Sep 17 15:39:42.960370 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.960329    2568 generic.go:356] "Generic (PLEG): container finished" podID="b8aadb40-1dda-4451-8765-ca03203e05a0" containerID="5fa9d7eade1fd9503450971ad4f16fee43df8b3fa83a88f1fb84f4d773f4f4ee" exitCode=0
Sep 17 15:39:42.961037 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.960408    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerDied","Data":"5fa9d7eade1fd9503450971ad4f16fee43df8b3fa83a88f1fb84f4d773f4f4ee"}
Sep 17 15:39:42.961112 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.961096    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:42.961186 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.961171    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:42.961241 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.961215    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:42.976541 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.976521    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:42.976916 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:42.976886    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:39:43.305678 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.305598    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9btfr"]
Sep 17 15:39:43.305830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.305768    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:43.305888 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:43.305870    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:43.307915 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.307885    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cgmvm"]
Sep 17 15:39:43.308027 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.308002    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:43.308147 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:43.308119    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:43.308490 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.308470    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b5xqc"]
Sep 17 15:39:43.308574 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.308557    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:43.308665 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:43.308645    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:43.964566 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.964533    2568 generic.go:356] "Generic (PLEG): container finished" podID="b8aadb40-1dda-4451-8765-ca03203e05a0" containerID="5b17abb7deef17eef1c6e2ef2e63515435ca9fa5fac0e2f5939dcaabb92a1171" exitCode=0
Sep 17 15:39:43.965236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:43.964616    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerDied","Data":"5b17abb7deef17eef1c6e2ef2e63515435ca9fa5fac0e2f5939dcaabb92a1171"}
Sep 17 15:39:44.825249 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:44.824966    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:44.825249 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:44.824966    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:44.825459 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:44.825282    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:44.825459 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:44.824969    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:44.825459 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:44.825353    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:44.825459 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:44.825425    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:46.825447 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:46.825416    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:46.825447 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:46.825441    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:46.825902 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:46.825429    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:46.825902 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:46.825543    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:46.825902 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:46.825617    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:46.825902 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:46.825685    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:48.825615 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:48.825589    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:48.826075 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:48.825591    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:48.826075 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:48.825688    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9btfr" podUID="437c724d-ac60-4db7-861a-704df11b37c9"
Sep 17 15:39:48.826075 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:48.825598    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:48.826075 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:48.825760    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgmvm" podUID="7ff164ba-34fa-45aa-b854-e48384d80808"
Sep 17 15:39:48.826075 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:48.825859    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b5xqc" podUID="a8098f77-1464-43c8-b25f-4991d2c44f90"
Sep 17 15:39:48.979338 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:48.979305    2568 generic.go:356] "Generic (PLEG): container finished" podID="b8aadb40-1dda-4451-8765-ca03203e05a0" containerID="fe1aab3a7cc60cb75e4bdf9641bdeff32e4acb39da79742a9965e1f66d22595b" exitCode=0
Sep 17 15:39:48.979494 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:48.979361    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerDied","Data":"fe1aab3a7cc60cb75e4bdf9641bdeff32e4acb39da79742a9965e1f66d22595b"}
Sep 17 15:39:49.212961 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.212936    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeReady"
Sep 17 15:39:49.213078 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.213067    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 15:39:49.262062 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.262033    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-llms2"]
Sep 17 15:39:49.268229 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.268206    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vsd8x"]
Sep 17 15:39:49.268347 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.268327    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.270552 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.270529    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 15:39:49.270660 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.270566    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2fbs9\""
Sep 17 15:39:49.270660 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.270566    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 15:39:49.272338 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.272321    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.274268 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.274249    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hgsvc\""
Sep 17 15:39:49.274370 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.274354    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 15:39:49.274426 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.274417    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 15:39:49.274481 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.274425    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 15:39:49.278110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.278087    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-llms2"]
Sep 17 15:39:49.278729 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.278680    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vsd8x"]
Sep 17 15:39:49.392964 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.392939    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc73331-de56-4a77-8391-bde40c51cae0-config-volume\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.393046 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.392988    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm5dm\" (UniqueName: \"kubernetes.io/projected/76cfc63e-e046-4228-aec2-8b884e290052-kube-api-access-qm5dm\") pod \"ingress-canary-vsd8x\" (UID: \"76cfc63e-e046-4228-aec2-8b884e290052\") " pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.393046 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.393010    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.393046 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.393028    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwpdf\" (UniqueName: \"kubernetes.io/projected/5fc73331-de56-4a77-8391-bde40c51cae0-kube-api-access-fwpdf\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.393140 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.393084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76cfc63e-e046-4228-aec2-8b884e290052-cert\") pod \"ingress-canary-vsd8x\" (UID: \"76cfc63e-e046-4228-aec2-8b884e290052\") " pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.393140 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.393114    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc73331-de56-4a77-8391-bde40c51cae0-tmp-dir\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.494227 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494196    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm5dm\" (UniqueName: \"kubernetes.io/projected/76cfc63e-e046-4228-aec2-8b884e290052-kube-api-access-qm5dm\") pod \"ingress-canary-vsd8x\" (UID: \"76cfc63e-e046-4228-aec2-8b884e290052\") " pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.494345 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494232    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.494345 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494259    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwpdf\" (UniqueName: \"kubernetes.io/projected/5fc73331-de56-4a77-8391-bde40c51cae0-kube-api-access-fwpdf\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.494345 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494277    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76cfc63e-e046-4228-aec2-8b884e290052-cert\") pod \"ingress-canary-vsd8x\" (UID: \"76cfc63e-e046-4228-aec2-8b884e290052\") " pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.494345 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494296    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc73331-de56-4a77-8391-bde40c51cae0-tmp-dir\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.494345 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494320    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:49.494586 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc73331-de56-4a77-8391-bde40c51cae0-config-volume\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.494586 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.494370    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:39:49.494586 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.494444    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls podName:5fc73331-de56-4a77-8391-bde40c51cae0 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:49.99442286 +0000 UTC m=+34.582467438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls") pod "dns-default-llms2" (UID: "5fc73331-de56-4a77-8391-bde40c51cae0") : secret "dns-default-metrics-tls" not found
Sep 17 15:39:49.494586 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.494486    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:49.494586 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.494545    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:21.494529723 +0000 UTC m=+66.082574302 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:39:49.494814 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc73331-de56-4a77-8391-bde40c51cae0-tmp-dir\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.494980 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.494956    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc73331-de56-4a77-8391-bde40c51cae0-config-volume\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.498830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.498707    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76cfc63e-e046-4228-aec2-8b884e290052-cert\") pod \"ingress-canary-vsd8x\" (UID: \"76cfc63e-e046-4228-aec2-8b884e290052\") " pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.505983 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.505958    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm5dm\" (UniqueName: \"kubernetes.io/projected/76cfc63e-e046-4228-aec2-8b884e290052-kube-api-access-qm5dm\") pod \"ingress-canary-vsd8x\" (UID: \"76cfc63e-e046-4228-aec2-8b884e290052\") " pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.506099 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.506083    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwpdf\" (UniqueName: \"kubernetes.io/projected/5fc73331-de56-4a77-8391-bde40c51cae0-kube-api-access-fwpdf\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.584101 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.584071    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vsd8x"
Sep 17 15:39:49.595367 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.595336    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:49.595498 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.595479    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:49.595553 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.595544    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret podName:437c724d-ac60-4db7-861a-704df11b37c9 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:21.595530246 +0000 UTC m=+66.183574811 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret") pod "global-pull-secret-syncer-9btfr" (UID: "437c724d-ac60-4db7-861a-704df11b37c9") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:39:49.696297 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.696269    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:49.696460 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.696444    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:39:49.696499 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.696464    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:39:49.696499 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.696474    2568 projected.go:196] Error preparing data for projected volume kube-api-access-rgvbq for pod openshift-network-diagnostics/network-check-target-b5xqc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:49.696575 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.696523    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq podName:a8098f77-1464-43c8-b25f-4991d2c44f90 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:21.696511525 +0000 UTC m=+66.284556095 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgvbq" (UniqueName: "kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq") pod "network-check-target-b5xqc" (UID: "a8098f77-1464-43c8-b25f-4991d2c44f90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:39:49.910202 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.910163    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vsd8x"]
Sep 17 15:39:49.912958 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:39:49.912931    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76cfc63e_e046_4228_aec2_8b884e290052.slice/crio-b7ba4a79088c6bcbac80470f2062cbc28d81682e84d67c189b8f1eb144946a96 WatchSource:0}: Error finding container b7ba4a79088c6bcbac80470f2062cbc28d81682e84d67c189b8f1eb144946a96: Status 404 returned error can't find the container with id b7ba4a79088c6bcbac80470f2062cbc28d81682e84d67c189b8f1eb144946a96
Sep 17 15:39:49.984746 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.984714    2568 generic.go:356] "Generic (PLEG): container finished" podID="b8aadb40-1dda-4451-8765-ca03203e05a0" containerID="6aa224c7b1513df645fafa0ed113b535d3ff58559485f65e1e5861367e41cc6b" exitCode=0
Sep 17 15:39:49.984917 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.984800    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerDied","Data":"6aa224c7b1513df645fafa0ed113b535d3ff58559485f65e1e5861367e41cc6b"}
Sep 17 15:39:49.985898 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.985875    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vsd8x" event={"ID":"76cfc63e-e046-4228-aec2-8b884e290052","Type":"ContainerStarted","Data":"b7ba4a79088c6bcbac80470f2062cbc28d81682e84d67c189b8f1eb144946a96"}
Sep 17 15:39:49.998392 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:49.998374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:49.998491 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.998478    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:39:49.998531 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:49.998521    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls podName:5fc73331-de56-4a77-8391-bde40c51cae0 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:50.998509757 +0000 UTC m=+35.586554323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls") pod "dns-default-llms2" (UID: "5fc73331-de56-4a77-8391-bde40c51cae0") : secret "dns-default-metrics-tls" not found
Sep 17 15:39:50.825655 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.825615    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:39:50.825828 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.825656    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:39:50.825828 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.825656    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:39:50.828730 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.828702    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 15:39:50.828857 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.828789    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ddz86\""
Sep 17 15:39:50.829580 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.829543    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 15:39:50.829580 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.829558    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7j2g9\""
Sep 17 15:39:50.829738 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.829629    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 15:39:50.829738 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.829558    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 15:39:50.992527 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:50.992493    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" event={"ID":"b8aadb40-1dda-4451-8765-ca03203e05a0","Type":"ContainerStarted","Data":"01fec1a313415d17cc66846d825b72b4f325930897bae1d3c7758532642cfe33"}
Sep 17 15:39:51.005997 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:51.005969    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:51.006129 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:51.006114    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:39:51.006183 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:51.006171    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls podName:5fc73331-de56-4a77-8391-bde40c51cae0 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:53.006153606 +0000 UTC m=+37.594198184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls") pod "dns-default-llms2" (UID: "5fc73331-de56-4a77-8391-bde40c51cae0") : secret "dns-default-metrics-tls" not found
Sep 17 15:39:51.025022 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:51.024957    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ql6ch" podStartSLOduration=8.422782759 podStartE2EDuration="36.024937467s" totalImagesPullingTime="27.602154708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:51.023741034 +0000 UTC m=+35.611785622" watchObservedRunningTime="2026-09-17 15:39:51.024937467 +0000 UTC m=+35.612982055"
Sep 17 15:39:52.998972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:52.998942    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vsd8x" event={"ID":"76cfc63e-e046-4228-aec2-8b884e290052","Type":"ContainerStarted","Data":"cd88b1442dd6eaead3874c814d0c04a72bbc8be30c28bc90c76ff190779b8544"}
Sep 17 15:39:53.021800 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:53.021774    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:53.021914 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:53.021901    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:39:53.021963 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:53.021954    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls podName:5fc73331-de56-4a77-8391-bde40c51cae0 nodeName:}" failed. No retries permitted until 2026-09-17 15:39:57.021936834 +0000 UTC m=+41.609981402 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls") pod "dns-default-llms2" (UID: "5fc73331-de56-4a77-8391-bde40c51cae0") : secret "dns-default-metrics-tls" not found
Sep 17 15:39:53.025731 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:53.025686    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vsd8x" podStartSLOduration=1.515977342 podStartE2EDuration="4.025670677s" totalImagesPullingTime="2.509693335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:39:53.025201199 +0000 UTC m=+37.613245787" watchObservedRunningTime="2026-09-17 15:39:53.025670677 +0000 UTC m=+37.613715407"
Sep 17 15:39:57.047855 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:39:57.047807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:39:57.048218 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:57.047958    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:39:57.048218 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:39:57.048016    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls podName:5fc73331-de56-4a77-8391-bde40c51cae0 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:05.048001782 +0000 UTC m=+49.636046351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls") pod "dns-default-llms2" (UID: "5fc73331-de56-4a77-8391-bde40c51cae0") : secret "dns-default-metrics-tls" not found
Sep 17 15:40:05.102403 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:05.102366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:40:05.102893 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:05.102509    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:40:05.102893 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:05.102568    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls podName:5fc73331-de56-4a77-8391-bde40c51cae0 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:21.10255145 +0000 UTC m=+65.690596032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls") pod "dns-default-llms2" (UID: "5fc73331-de56-4a77-8391-bde40c51cae0") : secret "dns-default-metrics-tls" not found
Sep 17 15:40:13.456094 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.456061    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"]
Sep 17 15:40:13.458821 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.458801    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.463762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.463735    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 15:40:13.463762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.463752    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lxb86\""
Sep 17 15:40:13.464104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.464049    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 17 15:40:13.464104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.464083    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 15:40:13.464104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.464053    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 17 15:40:13.471005 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.470427    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"]
Sep 17 15:40:13.558378 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.558337    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp76p\" (UniqueName: \"kubernetes.io/projected/949248cb-2674-43d6-8127-31928befc780-kube-api-access-pp76p\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.558560 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.558384    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/949248cb-2674-43d6-8127-31928befc780-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.558560 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.558445    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.562535 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.562510    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-d8g8b"]
Sep 17 15:40:13.564113 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.564099    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.566768 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.566746    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 17 15:40:13.566907 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.566746    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 17 15:40:13.567064 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.567048    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"]
Sep 17 15:40:13.567197 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.567180    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:40:13.567252 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.567217    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-96nbs\""
Sep 17 15:40:13.567482 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.567466    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 17 15:40:13.568646 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.568630    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8f69d7c84-59v6q"]
Sep 17 15:40:13.568765 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.568749    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.570256 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.570239    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.570912 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.570895    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7gg5h\""
Sep 17 15:40:13.573778 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.573754    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 17 15:40:13.573894 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.573857    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 17 15:40:13.573964 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.573912    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 17 15:40:13.576903 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.576882    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 15:40:13.577117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.577099    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 15:40:13.577194 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.577099    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 15:40:13.577257 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.577103    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:40:13.577314 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.577105    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kgfvz\""
Sep 17 15:40:13.577366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.577228    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 17 15:40:13.579421 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.579399    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-d8g8b"]
Sep 17 15:40:13.582605 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.582588    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 15:40:13.583714 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.583695    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"]
Sep 17 15:40:13.587546 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.587527    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f69d7c84-59v6q"]
Sep 17 15:40:13.659613 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659580    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-serving-cert\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.659613 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659612    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prlp5\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-kube-api-access-prlp5\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659632    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzcxd\" (UniqueName: \"kubernetes.io/projected/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-kube-api-access-fzcxd\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659672    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5b24329-b842-4a2e-b517-ae2adea120e5-serving-cert\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659703    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-image-registry-private-configuration\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659723    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-trusted-ca\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659750    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-trusted-ca\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82fc6885-94c5-4d96-888a-a71e58344ebb-ca-trust-extracted\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659786    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-certificates\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659803    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-bound-sa-token\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.659827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5b24329-b842-4a2e-b517-ae2adea120e5-config\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp76p\" (UniqueName: \"kubernetes.io/projected/949248cb-2674-43d6-8127-31928befc780-kube-api-access-pp76p\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659927    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/949248cb-2674-43d6-8127-31928befc780-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659944    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659960    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-config\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.659983    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.660079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-installation-pull-secrets\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.660167    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rcpz\" (UniqueName: \"kubernetes.io/projected/f5b24329-b842-4a2e-b517-ae2adea120e5-kube-api-access-8rcpz\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:13.660189    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:13.660244 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:13.660250    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls podName:949248cb-2674-43d6-8127-31928befc780 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:14.160237023 +0000 UTC m=+58.748281592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4kbch" (UID: "949248cb-2674-43d6-8127-31928befc780") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:13.660794 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.660772    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/949248cb-2674-43d6-8127-31928befc780-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.676170 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.676147    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp76p\" (UniqueName: \"kubernetes.io/projected/949248cb-2674-43d6-8127-31928befc780-kube-api-access-pp76p\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:13.761428 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761335    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzcxd\" (UniqueName: \"kubernetes.io/projected/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-kube-api-access-fzcxd\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.761428 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761384    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5b24329-b842-4a2e-b517-ae2adea120e5-serving-cert\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.761428 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761404    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-image-registry-private-configuration\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.761428 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-trusted-ca\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.761428 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761437    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-trusted-ca\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.761800 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761455    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82fc6885-94c5-4d96-888a-a71e58344ebb-ca-trust-extracted\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.761800 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761595    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-certificates\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.761800 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761651    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-bound-sa-token\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.761800 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761707    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5b24329-b842-4a2e-b517-ae2adea120e5-config\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.761800 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761785    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-config\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.762066 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761830    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.762066 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761889    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-installation-pull-secrets\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.762066 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761968    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82fc6885-94c5-4d96-888a-a71e58344ebb-ca-trust-extracted\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.762066 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.761974    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rcpz\" (UniqueName: \"kubernetes.io/projected/f5b24329-b842-4a2e-b517-ae2adea120e5-kube-api-access-8rcpz\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.762262 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.762086    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-serving-cert\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.762262 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.762121    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prlp5\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-kube-api-access-prlp5\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.762378 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.762344    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-certificates\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.763000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.762574    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5b24329-b842-4a2e-b517-ae2adea120e5-config\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.763000 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:13.762692    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:40:13.763000 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:13.762713    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f69d7c84-59v6q: secret "image-registry-tls" not found
Sep 17 15:40:13.763000 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:13.762788    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls podName:82fc6885-94c5-4d96-888a-a71e58344ebb nodeName:}" failed. No retries permitted until 2026-09-17 15:40:14.262768247 +0000 UTC m=+58.850812831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls") pod "image-registry-8f69d7c84-59v6q" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb") : secret "image-registry-tls" not found
Sep 17 15:40:13.763324 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.763098    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-config\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.763495 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.763468    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-trusted-ca\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.763764 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.763740    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-trusted-ca\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.764073 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.764053    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5b24329-b842-4a2e-b517-ae2adea120e5-serving-cert\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.764688 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.764661    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-image-registry-private-configuration\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.764816 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.764798    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-serving-cert\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.765113 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.765095    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-installation-pull-secrets\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.771554 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.771531    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzcxd\" (UniqueName: \"kubernetes.io/projected/8e244896-d0b1-4caf-ae7b-a31ea72fad9c-kube-api-access-fzcxd\") pod \"console-operator-696884fb9c-d8g8b\" (UID: \"8e244896-d0b1-4caf-ae7b-a31ea72fad9c\") " pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.776772 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.776742    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-bound-sa-token\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.776893 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.776778    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rcpz\" (UniqueName: \"kubernetes.io/projected/f5b24329-b842-4a2e-b517-ae2adea120e5-kube-api-access-8rcpz\") pod \"service-ca-operator-cf7c776b7-26xgf\" (UID: \"f5b24329-b842-4a2e-b517-ae2adea120e5\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:13.777032 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.777013    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prlp5\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-kube-api-access-prlp5\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:13.874506 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.874464    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:13.882066 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:13.882046    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"
Sep 17 15:40:14.152712 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:14.152558    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf"]
Sep 17 15:40:14.154933 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:14.154905    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-d8g8b"]
Sep 17 15:40:14.155722 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:14.155701    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b24329_b842_4a2e_b517_ae2adea120e5.slice/crio-a0a59bfa158d866ffc35363bbeed2f1c969bf1fd8f6c9c6c3e2ac0cd6fecc517 WatchSource:0}: Error finding container a0a59bfa158d866ffc35363bbeed2f1c969bf1fd8f6c9c6c3e2ac0cd6fecc517: Status 404 returned error can't find the container with id a0a59bfa158d866ffc35363bbeed2f1c969bf1fd8f6c9c6c3e2ac0cd6fecc517
Sep 17 15:40:14.157881 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:14.157861    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e244896_d0b1_4caf_ae7b_a31ea72fad9c.slice/crio-d1dc2bb68eae31a8d65808e16a70cacff2c2bc026015d038117a21f5aed1d875 WatchSource:0}: Error finding container d1dc2bb68eae31a8d65808e16a70cacff2c2bc026015d038117a21f5aed1d875: Status 404 returned error can't find the container with id d1dc2bb68eae31a8d65808e16a70cacff2c2bc026015d038117a21f5aed1d875
Sep 17 15:40:14.167346 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:14.167301    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:14.167474 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:14.167456    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:14.167522 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:14.167513    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls podName:949248cb-2674-43d6-8127-31928befc780 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:15.167498668 +0000 UTC m=+59.755543233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4kbch" (UID: "949248cb-2674-43d6-8127-31928befc780") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:14.267972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:14.267920    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:14.268147 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:14.268088    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:40:14.268147 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:14.268109    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f69d7c84-59v6q: secret "image-registry-tls" not found
Sep 17 15:40:14.268219 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:14.268164    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls podName:82fc6885-94c5-4d96-888a-a71e58344ebb nodeName:}" failed. No retries permitted until 2026-09-17 15:40:15.268149258 +0000 UTC m=+59.856193828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls") pod "image-registry-8f69d7c84-59v6q" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb") : secret "image-registry-tls" not found
Sep 17 15:40:14.979772 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:14.979738    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fm9p8"
Sep 17 15:40:15.054205 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:15.054167    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf" event={"ID":"f5b24329-b842-4a2e-b517-ae2adea120e5","Type":"ContainerStarted","Data":"a0a59bfa158d866ffc35363bbeed2f1c969bf1fd8f6c9c6c3e2ac0cd6fecc517"}
Sep 17 15:40:15.055754 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:15.055718    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" event={"ID":"8e244896-d0b1-4caf-ae7b-a31ea72fad9c","Type":"ContainerStarted","Data":"d1dc2bb68eae31a8d65808e16a70cacff2c2bc026015d038117a21f5aed1d875"}
Sep 17 15:40:15.176204 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:15.176132    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:15.176392 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:15.176371    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:15.176464 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:15.176448    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls podName:949248cb-2674-43d6-8127-31928befc780 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:17.176428051 +0000 UTC m=+61.764472619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4kbch" (UID: "949248cb-2674-43d6-8127-31928befc780") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:15.276975 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:15.276890    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:15.277119 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:15.277073    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:40:15.277119 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:15.277094    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f69d7c84-59v6q: secret "image-registry-tls" not found
Sep 17 15:40:15.277197 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:15.277165    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls podName:82fc6885-94c5-4d96-888a-a71e58344ebb nodeName:}" failed. No retries permitted until 2026-09-17 15:40:17.277146276 +0000 UTC m=+61.865190863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls") pod "image-registry-8f69d7c84-59v6q" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb") : secret "image-registry-tls" not found
Sep 17 15:40:17.063057 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.063014    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf" event={"ID":"f5b24329-b842-4a2e-b517-ae2adea120e5","Type":"ContainerStarted","Data":"d014684026a19f3d463de1936028f91f30d1a7f60a302de52f2d877247ff5388"}
Sep 17 15:40:17.064750 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.064727    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/0.log"
Sep 17 15:40:17.064880 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.064769    2568 generic.go:356] "Generic (PLEG): container finished" podID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c" containerID="001f0574a62fca07d57451df0bcf56b19106fb1eb021fecea62b5341000c3227" exitCode=255
Sep 17 15:40:17.064880 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.064799    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" event={"ID":"8e244896-d0b1-4caf-ae7b-a31ea72fad9c","Type":"ContainerDied","Data":"001f0574a62fca07d57451df0bcf56b19106fb1eb021fecea62b5341000c3227"}
Sep 17 15:40:17.065043 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.065027    2568 scope.go:117] "RemoveContainer" containerID="001f0574a62fca07d57451df0bcf56b19106fb1eb021fecea62b5341000c3227"
Sep 17 15:40:17.084978 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.084935    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf" podStartSLOduration=1.941421381 podStartE2EDuration="4.084918939s" totalImagesPullingTime="2.143497558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:17.08415212 +0000 UTC m=+61.672196710" watchObservedRunningTime="2026-09-17 15:40:17.084918939 +0000 UTC m=+61.672963525"
Sep 17 15:40:17.192514 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.192471    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:17.192787 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:17.192759    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:17.192906 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:17.192832    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls podName:949248cb-2674-43d6-8127-31928befc780 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:21.192810411 +0000 UTC m=+65.780854993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4kbch" (UID: "949248cb-2674-43d6-8127-31928befc780") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:17.293972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:17.293933    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:17.294142 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:17.294079    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:40:17.294142 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:17.294093    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f69d7c84-59v6q: secret "image-registry-tls" not found
Sep 17 15:40:17.294210 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:17.294158    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls podName:82fc6885-94c5-4d96-888a-a71e58344ebb nodeName:}" failed. No retries permitted until 2026-09-17 15:40:21.294142077 +0000 UTC m=+65.882186663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls") pod "image-registry-8f69d7c84-59v6q" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb") : secret "image-registry-tls" not found
Sep 17 15:40:18.068776 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.068748    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/1.log"
Sep 17 15:40:18.069228 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.069212    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/0.log"
Sep 17 15:40:18.069304 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.069248    2568 generic.go:356] "Generic (PLEG): container finished" podID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c" containerID="b876f2f5c9b8cffad85d44ab500f837c4dc634566e1ef8c4f1b7b95a42055c5b" exitCode=255
Sep 17 15:40:18.069371 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.069345    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" event={"ID":"8e244896-d0b1-4caf-ae7b-a31ea72fad9c","Type":"ContainerDied","Data":"b876f2f5c9b8cffad85d44ab500f837c4dc634566e1ef8c4f1b7b95a42055c5b"}
Sep 17 15:40:18.069441 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.069428    2568 scope.go:117] "RemoveContainer" containerID="001f0574a62fca07d57451df0bcf56b19106fb1eb021fecea62b5341000c3227"
Sep 17 15:40:18.069647 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.069622    2568 scope.go:117] "RemoveContainer" containerID="b876f2f5c9b8cffad85d44ab500f837c4dc634566e1ef8c4f1b7b95a42055c5b"
Sep 17 15:40:18.069828 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:18.069808    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-d8g8b_openshift-console-operator(8e244896-d0b1-4caf-ae7b-a31ea72fad9c)\"" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" podUID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c"
Sep 17 15:40:18.387291 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.387214    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v8264"]
Sep 17 15:40:18.396609 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.396585    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.400081 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.400057    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mv47d\""
Sep 17 15:40:18.400617 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.400424    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 15:40:18.400617 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.400464    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 15:40:18.400617 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.400486    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 15:40:18.400617 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.400428    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 15:40:18.402086 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.402065    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31e273df-5c65-481f-b2e7-c7bd56604e47-crio-socket\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.402171 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.402108    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.402171 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.402137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klmwb\" (UniqueName: \"kubernetes.io/projected/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-api-access-klmwb\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.402171 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.402163    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.402337 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.402303    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31e273df-5c65-481f-b2e7-c7bd56604e47-data-volume\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.408330 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.408310    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8264"]
Sep 17 15:40:18.502999 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.502949    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31e273df-5c65-481f-b2e7-c7bd56604e47-data-volume\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.503203 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.503033    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31e273df-5c65-481f-b2e7-c7bd56604e47-crio-socket\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.503203 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.503068    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.503203 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.503095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klmwb\" (UniqueName: \"kubernetes.io/projected/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-api-access-klmwb\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.503203 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.503116    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.503203 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.503129    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31e273df-5c65-481f-b2e7-c7bd56604e47-crio-socket\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.503412 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:18.503214    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:40:18.503412 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:18.503237    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:18.503412 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:18.503290    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:19.003270749 +0000 UTC m=+63.591315337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : configmap "kube-rbac-proxy" not found
Sep 17 15:40:18.503412 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:18.503308    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:19.003298654 +0000 UTC m=+63.591343219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:18.503412 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.503395    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31e273df-5c65-481f-b2e7-c7bd56604e47-data-volume\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:18.513272 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:18.513244    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klmwb\" (UniqueName: \"kubernetes.io/projected/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-api-access-klmwb\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:19.007355 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:19.007318    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:19.007509 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:19.007363    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:19.007509 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:19.007447    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:19.007509 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:19.007492    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:40:19.007611 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:19.007510    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:20.00749617 +0000 UTC m=+64.595540735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:19.007611 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:19.007542    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:20.007530762 +0000 UTC m=+64.595575326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : configmap "kube-rbac-proxy" not found
Sep 17 15:40:19.073166 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:19.073142    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/1.log"
Sep 17 15:40:19.073493 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:19.073446    2568 scope.go:117] "RemoveContainer" containerID="b876f2f5c9b8cffad85d44ab500f837c4dc634566e1ef8c4f1b7b95a42055c5b"
Sep 17 15:40:19.073614 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:19.073596    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-d8g8b_openshift-console-operator(8e244896-d0b1-4caf-ae7b-a31ea72fad9c)\"" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" podUID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c"
Sep 17 15:40:20.015159 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:20.015124    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:20.015159 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:20.015160    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:20.015348 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:20.015239    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:40:20.015348 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:20.015268    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:20.015348 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:20.015292    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:22.015278338 +0000 UTC m=+66.603322902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : configmap "kube-rbac-proxy" not found
Sep 17 15:40:20.015348 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:20.015321    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:22.015307187 +0000 UTC m=+66.603351752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:20.143673 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:20.143646    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2k5tc_eda22602-f2c3-4cab-8e46-4835749a5b97/dns-node-resolver/0.log"
Sep 17 15:40:21.122256 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.122224    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:40:21.122486 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.122359    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:40:21.122486 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.122412    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls podName:5fc73331-de56-4a77-8391-bde40c51cae0 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:53.122398928 +0000 UTC m=+97.710443497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls") pod "dns-default-llms2" (UID: "5fc73331-de56-4a77-8391-bde40c51cae0") : secret "dns-default-metrics-tls" not found
Sep 17 15:40:21.222959 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.222919    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:21.223320 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.223062    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:21.223320 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.223124    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls podName:949248cb-2674-43d6-8127-31928befc780 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:29.223110003 +0000 UTC m=+73.811154568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4kbch" (UID: "949248cb-2674-43d6-8127-31928befc780") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:21.323999 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.323965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:21.324173 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.324141    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:40:21.324173 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.324162    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f69d7c84-59v6q: secret "image-registry-tls" not found
Sep 17 15:40:21.324288 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.324226    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls podName:82fc6885-94c5-4d96-888a-a71e58344ebb nodeName:}" failed. No retries permitted until 2026-09-17 15:40:29.324206719 +0000 UTC m=+73.912251303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls") pod "image-registry-8f69d7c84-59v6q" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb") : secret "image-registry-tls" not found
Sep 17 15:40:21.526080 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.526041    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:40:21.528982 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.528961    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 15:40:21.536288 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.536268    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 15:40:21.536375 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:21.536332    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs podName:7ff164ba-34fa-45aa-b854-e48384d80808 nodeName:}" failed. No retries permitted until 2026-09-17 15:41:25.536313027 +0000 UTC m=+130.124357608 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs") pod "network-metrics-daemon-cgmvm" (UID: "7ff164ba-34fa-45aa-b854-e48384d80808") : secret "metrics-daemon-secret" not found
Sep 17 15:40:21.544474 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.544456    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kf4b5_1896bca3-c7f4-40bd-99c1-4a3f6d1a4365/node-ca/0.log"
Sep 17 15:40:21.627046 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.627015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:40:21.629910 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.629892    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 15:40:21.639327 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.639307    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/437c724d-ac60-4db7-861a-704df11b37c9-original-pull-secret\") pod \"global-pull-secret-syncer-9btfr\" (UID: \"437c724d-ac60-4db7-861a-704df11b37c9\") " pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:40:21.728388 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.728351    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:40:21.731245 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.731226    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 15:40:21.741831 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.741813    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 15:40:21.744809 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.744796    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9btfr"
Sep 17 15:40:21.752187 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:21.752166    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgvbq\" (UniqueName: \"kubernetes.io/projected/a8098f77-1464-43c8-b25f-4991d2c44f90-kube-api-access-rgvbq\") pod \"network-check-target-b5xqc\" (UID: \"a8098f77-1464-43c8-b25f-4991d2c44f90\") " pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:40:22.015332 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.015301    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9btfr"]
Sep 17 15:40:22.017966 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:22.017936    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437c724d_ac60_4db7_861a_704df11b37c9.slice/crio-687472318b1a2b30ad0d6f37b00e03d9f45e7dd3f83c11a434abd7bdc3e949ad WatchSource:0}: Error finding container 687472318b1a2b30ad0d6f37b00e03d9f45e7dd3f83c11a434abd7bdc3e949ad: Status 404 returned error can't find the container with id 687472318b1a2b30ad0d6f37b00e03d9f45e7dd3f83c11a434abd7bdc3e949ad
Sep 17 15:40:22.030446 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.030423    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:22.030558 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.030453    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:22.030621 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:22.030583    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:40:22.030621 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:22.030608    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:22.030750 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:22.030633    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:26.030617706 +0000 UTC m=+70.618662271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : configmap "kube-rbac-proxy" not found
Sep 17 15:40:22.030750 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:22.030672    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:26.030653996 +0000 UTC m=+70.618698574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:22.053117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.053064    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7j2g9\""
Sep 17 15:40:22.061091 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.061070    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:40:22.081855 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.081817    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9btfr" event={"ID":"437c724d-ac60-4db7-861a-704df11b37c9","Type":"ContainerStarted","Data":"687472318b1a2b30ad0d6f37b00e03d9f45e7dd3f83c11a434abd7bdc3e949ad"}
Sep 17 15:40:22.330226 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.330038    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b5xqc"]
Sep 17 15:40:22.332433 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:22.332408    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8098f77_1464_43c8_b25f_4991d2c44f90.slice/crio-551ee34222a8a822e2a672dca511ad890214e14e74ba162188e292916af2eec4 WatchSource:0}: Error finding container 551ee34222a8a822e2a672dca511ad890214e14e74ba162188e292916af2eec4: Status 404 returned error can't find the container with id 551ee34222a8a822e2a672dca511ad890214e14e74ba162188e292916af2eec4
Sep 17 15:40:22.343987 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:22.343971    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vsd8x_76cfc63e-e046-4228-aec2-8b884e290052/serve-healthcheck-canary/0.log"
Sep 17 15:40:23.086083 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:23.086042    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b5xqc" event={"ID":"a8098f77-1464-43c8-b25f-4991d2c44f90","Type":"ContainerStarted","Data":"551ee34222a8a822e2a672dca511ad890214e14e74ba162188e292916af2eec4"}
Sep 17 15:40:23.875123 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:23.875079    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:23.875594 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:23.875155    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:23.875594 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:23.875543    2568 scope.go:117] "RemoveContainer" containerID="b876f2f5c9b8cffad85d44ab500f837c4dc634566e1ef8c4f1b7b95a42055c5b"
Sep 17 15:40:23.875754 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:23.875727    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-d8g8b_openshift-console-operator(8e244896-d0b1-4caf-ae7b-a31ea72fad9c)\"" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" podUID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c"
Sep 17 15:40:26.064948 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:26.064906    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:26.065369 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:26.064955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:26.065369 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:26.065072    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:40:26.065369 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:26.065084    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:26.065369 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:26.065133    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:34.065113447 +0000 UTC m=+78.653158014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : configmap "kube-rbac-proxy" not found
Sep 17 15:40:26.065369 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:26.065152    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls podName:31e273df-5c65-481f-b2e7-c7bd56604e47 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:34.06514428 +0000 UTC m=+78.653188845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8264" (UID: "31e273df-5c65-481f-b2e7-c7bd56604e47") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:40:27.098210 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:27.098173    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9btfr" event={"ID":"437c724d-ac60-4db7-861a-704df11b37c9","Type":"ContainerStarted","Data":"eeb639b0dcd8e1098c6e71abdd1d2dec42c4698155c07316eddd64eaa8888b8f"}
Sep 17 15:40:27.099598 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:27.099579    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b5xqc" event={"ID":"a8098f77-1464-43c8-b25f-4991d2c44f90","Type":"ContainerStarted","Data":"d2fb425b4c864e5f19dcc353565bed60095d16ba03603bbe69f8623ad5b458e0"}
Sep 17 15:40:27.099723 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:27.099707    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:40:27.121872 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:27.121802    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9btfr" podStartSLOduration=66.558837012 podStartE2EDuration="1m11.121791494s" totalImagesPullingTime="4.562954482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:27.121525917 +0000 UTC m=+71.709570505" watchObservedRunningTime="2026-09-17 15:40:27.121791494 +0000 UTC m=+71.709836081"
Sep 17 15:40:27.144410 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:27.144371    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b5xqc" podStartSLOduration=67.900106434 podStartE2EDuration="1m12.144359007s" totalImagesPullingTime="4.244252573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:27.143288582 +0000 UTC m=+71.731333169" watchObservedRunningTime="2026-09-17 15:40:27.144359007 +0000 UTC m=+71.732403639"
Sep 17 15:40:29.290468 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:29.290429    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:29.290895 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:29.290582    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:29.290895 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:29.290646    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls podName:949248cb-2674-43d6-8127-31928befc780 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:45.290632264 +0000 UTC m=+89.878676842 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4kbch" (UID: "949248cb-2674-43d6-8127-31928befc780") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:40:29.391382 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:29.391352    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:29.393756 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:29.393732    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"image-registry-8f69d7c84-59v6q\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") " pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:29.489472 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:29.489444    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kgfvz\""
Sep 17 15:40:29.497420 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:29.497400    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:29.772192 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:29.772158    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f69d7c84-59v6q"]
Sep 17 15:40:29.775015 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:29.774984    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fc6885_94c5_4d96_888a_a71e58344ebb.slice/crio-a19b99c9c955d3ecac34d43aa1ea25f5eccb60d41599111d3d51e3870f86afa6 WatchSource:0}: Error finding container a19b99c9c955d3ecac34d43aa1ea25f5eccb60d41599111d3d51e3870f86afa6: Status 404 returned error can't find the container with id a19b99c9c955d3ecac34d43aa1ea25f5eccb60d41599111d3d51e3870f86afa6
Sep 17 15:40:30.108753 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:30.108670    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q" event={"ID":"82fc6885-94c5-4d96-888a-a71e58344ebb","Type":"ContainerStarted","Data":"1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31"}
Sep 17 15:40:30.108753 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:30.108726    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q" event={"ID":"82fc6885-94c5-4d96-888a-a71e58344ebb","Type":"ContainerStarted","Data":"a19b99c9c955d3ecac34d43aa1ea25f5eccb60d41599111d3d51e3870f86afa6"}
Sep 17 15:40:30.108976 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:30.108796    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:30.134051 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:30.133998    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q" podStartSLOduration=17.133960289 podStartE2EDuration="17.133960289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:30.133279832 +0000 UTC m=+74.721324432" watchObservedRunningTime="2026-09-17 15:40:30.133960289 +0000 UTC m=+74.722004876"
Sep 17 15:40:34.129570 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:34.129538    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:34.129570 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:34.129573    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:34.130061 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:34.130042    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31e273df-5c65-481f-b2e7-c7bd56604e47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:34.131875 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:34.131827    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31e273df-5c65-481f-b2e7-c7bd56604e47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8264\" (UID: \"31e273df-5c65-481f-b2e7-c7bd56604e47\") " pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:34.304866 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:34.304819    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8264"
Sep 17 15:40:34.583652 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:34.583603    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8264"]
Sep 17 15:40:34.585741 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:34.585713    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e273df_5c65_481f_b2e7_c7bd56604e47.slice/crio-acab62c2d99663a440ab03e78cc54399a1afddd95829e48277a9bb8535d964bd WatchSource:0}: Error finding container acab62c2d99663a440ab03e78cc54399a1afddd95829e48277a9bb8535d964bd: Status 404 returned error can't find the container with id acab62c2d99663a440ab03e78cc54399a1afddd95829e48277a9bb8535d964bd
Sep 17 15:40:35.125200 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:35.125165    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8264" event={"ID":"31e273df-5c65-481f-b2e7-c7bd56604e47","Type":"ContainerStarted","Data":"7513e58bf9fff23d711a64594d8657b29a0bf345b194c1f67ca47aacf4f3117c"}
Sep 17 15:40:35.125378 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:35.125227    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8264" event={"ID":"31e273df-5c65-481f-b2e7-c7bd56604e47","Type":"ContainerStarted","Data":"acab62c2d99663a440ab03e78cc54399a1afddd95829e48277a9bb8535d964bd"}
Sep 17 15:40:36.130414 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:36.130381    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8264" event={"ID":"31e273df-5c65-481f-b2e7-c7bd56604e47","Type":"ContainerStarted","Data":"66aa9ca90a5cbe2bf0082b3baa1a03a4db69129b92fc36b82d90847d3612b5b4"}
Sep 17 15:40:36.825473 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:36.825439    2568 scope.go:117] "RemoveContainer" containerID="b876f2f5c9b8cffad85d44ab500f837c4dc634566e1ef8c4f1b7b95a42055c5b"
Sep 17 15:40:37.134820 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.134750    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:40:37.135247 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.135230    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/1.log"
Sep 17 15:40:37.135289 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.135263    2568 generic.go:356] "Generic (PLEG): container finished" podID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c" containerID="98ff177dcd62bb757f74a7c944b2f042b1d192fc2e90795998830e01ac10a84b" exitCode=255
Sep 17 15:40:37.135339 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.135319    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" event={"ID":"8e244896-d0b1-4caf-ae7b-a31ea72fad9c","Type":"ContainerDied","Data":"98ff177dcd62bb757f74a7c944b2f042b1d192fc2e90795998830e01ac10a84b"}
Sep 17 15:40:37.135389 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.135378    2568 scope.go:117] "RemoveContainer" containerID="b876f2f5c9b8cffad85d44ab500f837c4dc634566e1ef8c4f1b7b95a42055c5b"
Sep 17 15:40:37.135696 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.135677    2568 scope.go:117] "RemoveContainer" containerID="98ff177dcd62bb757f74a7c944b2f042b1d192fc2e90795998830e01ac10a84b"
Sep 17 15:40:37.135916 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:37.135882    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-d8g8b_openshift-console-operator(8e244896-d0b1-4caf-ae7b-a31ea72fad9c)\"" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" podUID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c"
Sep 17 15:40:37.137473 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.137448    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8264" event={"ID":"31e273df-5c65-481f-b2e7-c7bd56604e47","Type":"ContainerStarted","Data":"84654a2fa0d0216bdf58101d62ff4c62bd1ac2149a61e98f08462ef8ba31617a"}
Sep 17 15:40:37.173707 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:37.173667    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v8264" podStartSLOduration=17.508287418 podStartE2EDuration="19.173654557s" totalImagesPullingTime="1.665367139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:40:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:37.172858563 +0000 UTC m=+81.760903151" watchObservedRunningTime="2026-09-17 15:40:37.173654557 +0000 UTC m=+81.761699143"
Sep 17 15:40:38.141758 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:38.141729    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:40:41.447673 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.447639    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"]
Sep 17 15:40:41.450645 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.450621    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.453741 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.453715    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4tjjk\""
Sep 17 15:40:41.454286 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.454264    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 17 15:40:41.454620 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.454599    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 17 15:40:41.465117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.465096    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"]
Sep 17 15:40:41.466876 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.466861    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.469732 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.469711    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 15:40:41.472373 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.472354    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 15:40:41.472506 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.472484    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 15:40:41.472656 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.472463    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5w5wm\""
Sep 17 15:40:41.472779 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.472436    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 17 15:40:41.473691 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.473670    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"]
Sep 17 15:40:41.497686 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.496097    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"]
Sep 17 15:40:41.507318 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.507291    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8f69d7c84-59v6q"]
Sep 17 15:40:41.583928 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.583890    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wmgm\" (UniqueName: \"kubernetes.io/projected/ea3252a9-1c16-4fd9-8cb9-c029da866add-kube-api-access-6wmgm\") pod \"managed-serviceaccount-addon-agent-5bffcbff94-5twk7\" (UID: \"ea3252a9-1c16-4fd9-8cb9-c029da866add\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.584119 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.583973    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6bc73e9d-9065-4049-8079-59715503f460-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-snzd6\" (UID: \"6bc73e9d-9065-4049-8079-59715503f460\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.584119 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.584024    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea3252a9-1c16-4fd9-8cb9-c029da866add-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bffcbff94-5twk7\" (UID: \"ea3252a9-1c16-4fd9-8cb9-c029da866add\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.584208 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.584128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6bc73e9d-9065-4049-8079-59715503f460-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-snzd6\" (UID: \"6bc73e9d-9065-4049-8079-59715503f460\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.591385 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.591352    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6df7765dc8-85gxv"]
Sep 17 15:40:41.593669 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.593642    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.625236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.625206    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6df7765dc8-85gxv"]
Sep 17 15:40:41.684787 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684751    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6bc73e9d-9065-4049-8079-59715503f460-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-snzd6\" (UID: \"6bc73e9d-9065-4049-8079-59715503f460\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684796    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00e78b9c-d2cf-417a-a538-05623d7b1398-installation-pull-secrets\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684823    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67wtq\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-kube-api-access-67wtq\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea3252a9-1c16-4fd9-8cb9-c029da866add-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bffcbff94-5twk7\" (UID: \"ea3252a9-1c16-4fd9-8cb9-c029da866add\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684894    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00e78b9c-d2cf-417a-a538-05623d7b1398-image-registry-private-configuration\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684929    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00e78b9c-d2cf-417a-a538-05623d7b1398-ca-trust-extracted\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00e78b9c-d2cf-417a-a538-05623d7b1398-registry-certificates\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.684965    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-bound-sa-token\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685004 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.685001    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6bc73e9d-9065-4049-8079-59715503f460-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-snzd6\" (UID: \"6bc73e9d-9065-4049-8079-59715503f460\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.685393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.685026    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-registry-tls\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.685058    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00e78b9c-d2cf-417a-a538-05623d7b1398-trusted-ca\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.685393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.685095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wmgm\" (UniqueName: \"kubernetes.io/projected/ea3252a9-1c16-4fd9-8cb9-c029da866add-kube-api-access-6wmgm\") pod \"managed-serviceaccount-addon-agent-5bffcbff94-5twk7\" (UID: \"ea3252a9-1c16-4fd9-8cb9-c029da866add\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.685495 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.685428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6bc73e9d-9065-4049-8079-59715503f460-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-snzd6\" (UID: \"6bc73e9d-9065-4049-8079-59715503f460\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.687391 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.687368    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea3252a9-1c16-4fd9-8cb9-c029da866add-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bffcbff94-5twk7\" (UID: \"ea3252a9-1c16-4fd9-8cb9-c029da866add\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.687490 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.687372    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6bc73e9d-9065-4049-8079-59715503f460-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-snzd6\" (UID: \"6bc73e9d-9065-4049-8079-59715503f460\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.704320 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.704263    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wmgm\" (UniqueName: \"kubernetes.io/projected/ea3252a9-1c16-4fd9-8cb9-c029da866add-kube-api-access-6wmgm\") pod \"managed-serviceaccount-addon-agent-5bffcbff94-5twk7\" (UID: \"ea3252a9-1c16-4fd9-8cb9-c029da866add\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.759566 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.759520    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"
Sep 17 15:40:41.783042 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.783012    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"
Sep 17 15:40:41.789519 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789477    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00e78b9c-d2cf-417a-a538-05623d7b1398-installation-pull-secrets\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.789632 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789525    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67wtq\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-kube-api-access-67wtq\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.789632 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789559    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00e78b9c-d2cf-417a-a538-05623d7b1398-image-registry-private-configuration\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.789632 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789587    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00e78b9c-d2cf-417a-a538-05623d7b1398-ca-trust-extracted\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.789632 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789613    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00e78b9c-d2cf-417a-a538-05623d7b1398-registry-certificates\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.789785 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789638    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-bound-sa-token\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.789785 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789676    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-registry-tls\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.789785 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.789698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00e78b9c-d2cf-417a-a538-05623d7b1398-trusted-ca\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.790550 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.790500    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00e78b9c-d2cf-417a-a538-05623d7b1398-ca-trust-extracted\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.791016 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.790996    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00e78b9c-d2cf-417a-a538-05623d7b1398-registry-certificates\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.791417 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.791390    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00e78b9c-d2cf-417a-a538-05623d7b1398-trusted-ca\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.792465 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.792420    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00e78b9c-d2cf-417a-a538-05623d7b1398-image-registry-private-configuration\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.793082 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.792965    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00e78b9c-d2cf-417a-a538-05623d7b1398-installation-pull-secrets\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.793186 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.793161    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-registry-tls\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.801317 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.801290    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67wtq\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-kube-api-access-67wtq\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.801657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.801634    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00e78b9c-d2cf-417a-a538-05623d7b1398-bound-sa-token\") pod \"image-registry-6df7765dc8-85gxv\" (UID: \"00e78b9c-d2cf-417a-a538-05623d7b1398\") " pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:41.902772 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:41.902734    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:42.036546 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:42.036468    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6"]
Sep 17 15:40:42.041363 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:42.041337    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc73e9d_9065_4049_8079_59715503f460.slice/crio-279a5383f8e5eb42098b4a54bfbebbbc534c181e80807d0ff0ef667ef44340c0 WatchSource:0}: Error finding container 279a5383f8e5eb42098b4a54bfbebbbc534c181e80807d0ff0ef667ef44340c0: Status 404 returned error can't find the container with id 279a5383f8e5eb42098b4a54bfbebbbc534c181e80807d0ff0ef667ef44340c0
Sep 17 15:40:42.061791 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:42.061767    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7"]
Sep 17 15:40:42.063817 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:42.063790    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea3252a9_1c16_4fd9_8cb9_c029da866add.slice/crio-4bed4588f734041c49ac3514af98a6b2b892e57090095dc2463041541b021a91 WatchSource:0}: Error finding container 4bed4588f734041c49ac3514af98a6b2b892e57090095dc2463041541b021a91: Status 404 returned error can't find the container with id 4bed4588f734041c49ac3514af98a6b2b892e57090095dc2463041541b021a91
Sep 17 15:40:42.155915 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:42.155874    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6" event={"ID":"6bc73e9d-9065-4049-8079-59715503f460","Type":"ContainerStarted","Data":"279a5383f8e5eb42098b4a54bfbebbbc534c181e80807d0ff0ef667ef44340c0"}
Sep 17 15:40:42.156916 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:42.156890    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7" event={"ID":"ea3252a9-1c16-4fd9-8cb9-c029da866add","Type":"ContainerStarted","Data":"4bed4588f734041c49ac3514af98a6b2b892e57090095dc2463041541b021a91"}
Sep 17 15:40:42.177805 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:42.177780    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6df7765dc8-85gxv"]
Sep 17 15:40:42.180796 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:42.180772    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e78b9c_d2cf_417a_a538_05623d7b1398.slice/crio-a149acf040bef843980486e1cad996a15cd86c9082949d76bd7c0dfb5e40d6a2 WatchSource:0}: Error finding container a149acf040bef843980486e1cad996a15cd86c9082949d76bd7c0dfb5e40d6a2: Status 404 returned error can't find the container with id a149acf040bef843980486e1cad996a15cd86c9082949d76bd7c0dfb5e40d6a2
Sep 17 15:40:43.163031 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:43.162983    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6df7765dc8-85gxv" event={"ID":"00e78b9c-d2cf-417a-a538-05623d7b1398","Type":"ContainerStarted","Data":"976d05359f738da44cecc3c80fbe371d18ce3f426381b75fede1909fc81c273e"}
Sep 17 15:40:43.163489 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:43.163048    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6df7765dc8-85gxv" event={"ID":"00e78b9c-d2cf-417a-a538-05623d7b1398","Type":"ContainerStarted","Data":"a149acf040bef843980486e1cad996a15cd86c9082949d76bd7c0dfb5e40d6a2"}
Sep 17 15:40:43.163710 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:43.163656    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:40:43.203307 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:43.203257    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6df7765dc8-85gxv" podStartSLOduration=2.2032415260000002 podStartE2EDuration="2.203241526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:43.202714091 +0000 UTC m=+87.790758680" watchObservedRunningTime="2026-09-17 15:40:43.203241526 +0000 UTC m=+87.791286113"
Sep 17 15:40:43.875439 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:43.875409    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:43.875647 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:43.875462    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:40:43.875817 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:43.875798    2568 scope.go:117] "RemoveContainer" containerID="98ff177dcd62bb757f74a7c944b2f042b1d192fc2e90795998830e01ac10a84b"
Sep 17 15:40:43.875986 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:43.875966    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-d8g8b_openshift-console-operator(8e244896-d0b1-4caf-ae7b-a31ea72fad9c)\"" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" podUID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c"
Sep 17 15:40:44.168218 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:44.168123    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6" event={"ID":"6bc73e9d-9065-4049-8079-59715503f460","Type":"ContainerStarted","Data":"4c602bb885b49e3a7d795d85f775a298ee84eba507d12c4a2ddf4c0d5b8136d8"}
Sep 17 15:40:44.192338 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:44.192288    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-snzd6" podStartSLOduration=2.03869653 podStartE2EDuration="3.192270786s" totalImagesPullingTime="1.153574256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:44.191775297 +0000 UTC m=+88.779819884" watchObservedRunningTime="2026-09-17 15:40:44.192270786 +0000 UTC m=+88.780315374"
Sep 17 15:40:45.176793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:45.176753    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7" event={"ID":"ea3252a9-1c16-4fd9-8cb9-c029da866add","Type":"ContainerStarted","Data":"be18fee7efd82d6ed8c9b060745f484f6f814eb98c5171918f5ba600adb0c20e"}
Sep 17 15:40:45.204562 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:45.204514    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bffcbff94-5twk7" podStartSLOduration=1.8868393110000001 podStartE2EDuration="4.204500122s" totalImagesPullingTime="2.317660811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:45.20415801 +0000 UTC m=+89.792202598" watchObservedRunningTime="2026-09-17 15:40:45.204500122 +0000 UTC m=+89.792544709"
Sep 17 15:40:45.320682 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:45.320632    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:45.323176 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:45.323149    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/949248cb-2674-43d6-8127-31928befc780-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4kbch\" (UID: \"949248cb-2674-43d6-8127-31928befc780\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:45.569922 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:45.569890    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lxb86\""
Sep 17 15:40:45.578128 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:45.578096    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"
Sep 17 15:40:45.852386 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:45.852300    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch"]
Sep 17 15:40:45.855317 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:45.855281    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949248cb_2674_43d6_8127_31928befc780.slice/crio-9403dc786c7f73ba81ea222267a95e6ab769fc8aaffa30e37253bea68a4a4b27 WatchSource:0}: Error finding container 9403dc786c7f73ba81ea222267a95e6ab769fc8aaffa30e37253bea68a4a4b27: Status 404 returned error can't find the container with id 9403dc786c7f73ba81ea222267a95e6ab769fc8aaffa30e37253bea68a4a4b27
Sep 17 15:40:46.180585 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:46.180499    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch" event={"ID":"949248cb-2674-43d6-8127-31928befc780","Type":"ContainerStarted","Data":"9403dc786c7f73ba81ea222267a95e6ab769fc8aaffa30e37253bea68a4a4b27"}
Sep 17 15:40:48.192767 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:48.192727    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch" event={"ID":"949248cb-2674-43d6-8127-31928befc780","Type":"ContainerStarted","Data":"3df1d075c02b0131628f83571488edfa771d9651287baea8d379601ddee5863f"}
Sep 17 15:40:48.223299 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:48.223245    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4kbch" podStartSLOduration=33.490365263 podStartE2EDuration="35.223230377s" totalImagesPullingTime="1.732865114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:48.215511438 +0000 UTC m=+92.803556036" watchObservedRunningTime="2026-09-17 15:40:48.223230377 +0000 UTC m=+92.811274963"
Sep 17 15:40:51.513037 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:51.513010    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:40:53.177795 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:53.177715    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:40:53.179916 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:53.179892    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fc73331-de56-4a77-8391-bde40c51cae0-metrics-tls\") pod \"dns-default-llms2\" (UID: \"5fc73331-de56-4a77-8391-bde40c51cae0\") " pod="openshift-dns/dns-default-llms2"
Sep 17 15:40:53.481188 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:53.481153    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2fbs9\""
Sep 17 15:40:53.489353 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:53.489331    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-llms2"
Sep 17 15:40:53.771185 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:53.771165    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-llms2"]
Sep 17 15:40:53.773548 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:53.773510    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc73331_de56_4a77_8391_bde40c51cae0.slice/crio-ff9791b34124075759f244df496355a381acb31e17c6f0cde7eac49e1b6f11c3 WatchSource:0}: Error finding container ff9791b34124075759f244df496355a381acb31e17c6f0cde7eac49e1b6f11c3: Status 404 returned error can't find the container with id ff9791b34124075759f244df496355a381acb31e17c6f0cde7eac49e1b6f11c3
Sep 17 15:40:54.216154 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:54.216119    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llms2" event={"ID":"5fc73331-de56-4a77-8391-bde40c51cae0","Type":"ContainerStarted","Data":"ff9791b34124075759f244df496355a381acb31e17c6f0cde7eac49e1b6f11c3"}
Sep 17 15:40:55.608959 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.608927    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"]
Sep 17 15:40:55.611143 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.611127    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.614932 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.614905    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 15:40:55.615532 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.615512    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 15:40:55.615687 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.615669    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 15:40:55.616196 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.616177    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kn9t6\""
Sep 17 15:40:55.624424 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.624401    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"]
Sep 17 15:40:55.633615 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.633594    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"]
Sep 17 15:40:55.635587 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.635570    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.640441 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.640421    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 17 15:40:55.640441 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.640436    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 17 15:40:55.640598 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.640436    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5k4ct\""
Sep 17 15:40:55.640651 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.640603    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 17 15:40:55.643338 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.643307    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pwdf8"]
Sep 17 15:40:55.645287 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.645273    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.647659 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.647642    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dl7rm\""
Sep 17 15:40:55.647786 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.647771    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 15:40:55.648010 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.647994    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 15:40:55.648216 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.648199    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 15:40:55.657033 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.657014    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"]
Sep 17 15:40:55.696579 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.696547    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffa61525-735b-40bd-89ea-ccf07b290ab7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.696722 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.696586    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffa61525-735b-40bd-89ea-ccf07b290ab7-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.696722 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.696690    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8k28\" (UniqueName: \"kubernetes.io/projected/ffa61525-735b-40bd-89ea-ccf07b290ab7-kube-api-access-d8k28\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.696722 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.696719    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffa61525-735b-40bd-89ea-ccf07b290ab7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.797186 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797153    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffa61525-735b-40bd-89ea-ccf07b290ab7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.797186 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797190    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-root\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797211    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqgwl\" (UniqueName: \"kubernetes.io/projected/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-api-access-vqgwl\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.797394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797245    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-wtmp\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797275    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-tls\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797298    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-metrics-client-ca\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797321    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-textfile\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797341    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r4qr\" (UniqueName: \"kubernetes.io/projected/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-kube-api-access-8r4qr\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797366    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffa61525-735b-40bd-89ea-ccf07b290ab7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797448    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-sys\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffa61525-735b-40bd-89ea-ccf07b290ab7-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797512    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797592    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b870e596-5e4e-41ba-9c45-d694ab3d50a5-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797616    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797631    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870e596-5e4e-41ba-9c45-d694ab3d50a5-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797670    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.797726 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.797724    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8k28\" (UniqueName: \"kubernetes.io/projected/ffa61525-735b-40bd-89ea-ccf07b290ab7-kube-api-access-d8k28\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.798233 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.798212    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffa61525-735b-40bd-89ea-ccf07b290ab7-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.799715 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.799692    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffa61525-735b-40bd-89ea-ccf07b290ab7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.799908 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.799888    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffa61525-735b-40bd-89ea-ccf07b290ab7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.813207 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.813185    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8k28\" (UniqueName: \"kubernetes.io/projected/ffa61525-735b-40bd-89ea-ccf07b290ab7-kube-api-access-d8k28\") pod \"openshift-state-metrics-67dcf8c979-hcvxd\" (UID: \"ffa61525-735b-40bd-89ea-ccf07b290ab7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.827155 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.827131    2568 scope.go:117] "RemoveContainer" containerID="98ff177dcd62bb757f74a7c944b2f042b1d192fc2e90795998830e01ac10a84b"
Sep 17 15:40:55.827310 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:55.827288    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-d8g8b_openshift-console-operator(8e244896-d0b1-4caf-ae7b-a31ea72fad9c)\"" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" podUID="8e244896-d0b1-4caf-ae7b-a31ea72fad9c"
Sep 17 15:40:55.898534 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898459    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r4qr\" (UniqueName: \"kubernetes.io/projected/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-kube-api-access-8r4qr\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.898534 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898497    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.898739 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898535    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.898739 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898563    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-sys\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.898739 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898611    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.898739 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898668    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-sys\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.898739 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898677    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b870e596-5e4e-41ba-9c45-d694ab3d50a5-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.899019 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898761    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.899019 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.898786    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870e596-5e4e-41ba-9c45-d694ab3d50a5-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.899019 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:55.898938    2568 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 17 15:40:55.899019 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:55.898997    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-tls podName:b870e596-5e4e-41ba-9c45-d694ab3d50a5 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:56.398977476 +0000 UTC m=+100.987022063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-gkmrb" (UID: "b870e596-5e4e-41ba-9c45-d694ab3d50a5") : secret "kube-state-metrics-tls" not found
Sep 17 15:40:55.899019 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899009    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b870e596-5e4e-41ba-9c45-d694ab3d50a5-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.899366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899339    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.899551 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899464    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.899551 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899537    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-root\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.899719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899575    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-root\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.899719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899580    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqgwl\" (UniqueName: \"kubernetes.io/projected/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-api-access-vqgwl\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.899719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-wtmp\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.899719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-tls\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.899719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899685    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-metrics-client-ca\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.899719 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899711    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-textfile\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.900043 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899792    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-wtmp\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.900043 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.899868    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.900043 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.900016    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-textfile\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.900183 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.900130    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870e596-5e4e-41ba-9c45-d694ab3d50a5-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.900313 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.900291    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-metrics-client-ca\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.901565 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.901538    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.901666 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.901568    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.902163 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.902146    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-node-exporter-tls\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.911192 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.911165    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqgwl\" (UniqueName: \"kubernetes.io/projected/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-api-access-vqgwl\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:55.911281 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.911200    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r4qr\" (UniqueName: \"kubernetes.io/projected/8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf-kube-api-access-8r4qr\") pod \"node-exporter-pwdf8\" (UID: \"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf\") " pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.920137 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.920123    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"
Sep 17 15:40:55.953127 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:55.953099    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwdf8"
Sep 17 15:40:55.963781 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:55.963524    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e8de3e9_4f0e_4da1_8955_99fce6ec9bcf.slice/crio-e300a562e57943404dfba64fecd2b63ff1cb4100eeab355ca5a025b12d3f482b WatchSource:0}: Error finding container e300a562e57943404dfba64fecd2b63ff1cb4100eeab355ca5a025b12d3f482b: Status 404 returned error can't find the container with id e300a562e57943404dfba64fecd2b63ff1cb4100eeab355ca5a025b12d3f482b
Sep 17 15:40:56.194539 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.194512    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd"]
Sep 17 15:40:56.196950 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:56.196917    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa61525_735b_40bd_89ea_ccf07b290ab7.slice/crio-1ab37c5946628d7b827f96513f57c3d6246d9be9837272347c8a76c84672b10c WatchSource:0}: Error finding container 1ab37c5946628d7b827f96513f57c3d6246d9be9837272347c8a76c84672b10c: Status 404 returned error can't find the container with id 1ab37c5946628d7b827f96513f57c3d6246d9be9837272347c8a76c84672b10c
Sep 17 15:40:56.223198 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.223172    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdf8" event={"ID":"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf","Type":"ContainerStarted","Data":"e300a562e57943404dfba64fecd2b63ff1cb4100eeab355ca5a025b12d3f482b"}
Sep 17 15:40:56.224899 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.224874    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llms2" event={"ID":"5fc73331-de56-4a77-8391-bde40c51cae0","Type":"ContainerStarted","Data":"ed41ac5ef70121d79a7e85f0a5566da7de1db86fb3f07c82ad9489dee7b5a7e4"}
Sep 17 15:40:56.224995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.224924    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-llms2" event={"ID":"5fc73331-de56-4a77-8391-bde40c51cae0","Type":"ContainerStarted","Data":"c76a80faf99c69857b90f2896f0853a79cbf59748da48f36ef269f9d742a5ffd"}
Sep 17 15:40:56.225039 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.225001    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-llms2"
Sep 17 15:40:56.225996 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.225978    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd" event={"ID":"ffa61525-735b-40bd-89ea-ccf07b290ab7","Type":"ContainerStarted","Data":"1ab37c5946628d7b827f96513f57c3d6246d9be9837272347c8a76c84672b10c"}
Sep 17 15:40:56.251431 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.251383    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-llms2" podStartSLOduration=65.865940243 podStartE2EDuration="1m7.251368484s" totalImagesPullingTime="1.385428241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:56.250964775 +0000 UTC m=+100.839009363" watchObservedRunningTime="2026-09-17 15:40:56.251368484 +0000 UTC m=+100.839413070"
Sep 17 15:40:56.403527 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.403494    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:56.406361 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.406332    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b870e596-5e4e-41ba-9c45-d694ab3d50a5-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-gkmrb\" (UID: \"b870e596-5e4e-41ba-9c45-d694ab3d50a5\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:56.544266 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.544186    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"
Sep 17 15:40:56.732234 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.732201    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:40:56.736061 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.736031    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.740905 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.740881    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 15:40:56.741021 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.740985    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h9nm4\""
Sep 17 15:40:56.741084 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.741042    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 15:40:56.741141 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.741088    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 15:40:56.741193 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.741157    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 15:40:56.741391 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.741374    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 15:40:56.741474 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.741416    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 15:40:56.741592 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.741574    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 15:40:56.741656 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.741607    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 15:40:56.747146 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.747131    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 15:40:56.756479 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.756379    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:40:56.808498 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808429    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5c6x\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-kube-api-access-t5c6x\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808498 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-out\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808693 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808524    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808693 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808591    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808693 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808870 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808690    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808870 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808727    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-web-config\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808870 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808748    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808870 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808870 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808809    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.808870 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808832    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.809160 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808913    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.809160 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.808956    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.837872 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.837850    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb"]
Sep 17 15:40:56.841813 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:56.841785    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb870e596_5e4e_41ba_9c45_d694ab3d50a5.slice/crio-554f268387d5df177b0bb8a8ff2c1f6bf3d2a3387aaba1dd0aa0d290aa7d9cb3 WatchSource:0}: Error finding container 554f268387d5df177b0bb8a8ff2c1f6bf3d2a3387aaba1dd0aa0d290aa7d9cb3: Status 404 returned error can't find the container with id 554f268387d5df177b0bb8a8ff2c1f6bf3d2a3387aaba1dd0aa0d290aa7d9cb3
Sep 17 15:40:56.909427 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909405    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909427 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909433    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909594 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909461    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5c6x\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-kube-api-access-t5c6x\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909594 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909494    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-out\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909594 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909531    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909594 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909561    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909613    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909635    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909657    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-web-config\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909680    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909706    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.909793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.909772    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.910167 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:40:56.909945    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle podName:4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2 nodeName:}" failed. No retries permitted until 2026-09-17 15:40:57.409925331 +0000 UTC m=+101.997969907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2") : configmap references non-existent config key: ca-bundle.crt
Sep 17 15:40:56.911618 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.911277    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.911618 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.911552    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.913328 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.913200    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.913328 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.913288    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.913649 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.913628    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.913880 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.913858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-out\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.914053 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.914036    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.914324 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.914304    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.914468 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.914452    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.914789 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.914769    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-web-config\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.915290 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.915275    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:56.927993 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:56.927968    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5c6x\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-kube-api-access-t5c6x\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:57.232553 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.232514    2568 generic.go:356] "Generic (PLEG): container finished" podID="8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf" containerID="a4cb5531c2eafb46a6b7943a3d1f419dfaa44a22cdfb258abbd72c86480de57f" exitCode=0
Sep 17 15:40:57.232700 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.232595    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdf8" event={"ID":"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf","Type":"ContainerDied","Data":"a4cb5531c2eafb46a6b7943a3d1f419dfaa44a22cdfb258abbd72c86480de57f"}
Sep 17 15:40:57.234620 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.234590    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb" event={"ID":"b870e596-5e4e-41ba-9c45-d694ab3d50a5","Type":"ContainerStarted","Data":"554f268387d5df177b0bb8a8ff2c1f6bf3d2a3387aaba1dd0aa0d290aa7d9cb3"}
Sep 17 15:40:57.236867 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.236822    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd" event={"ID":"ffa61525-735b-40bd-89ea-ccf07b290ab7","Type":"ContainerStarted","Data":"2dfbe04e85436980a337c1336ed87932317065b10a873232f52ef0cb76633c10"}
Sep 17 15:40:57.236979 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.236886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd" event={"ID":"ffa61525-735b-40bd-89ea-ccf07b290ab7","Type":"ContainerStarted","Data":"cd8e9c9ea195e378b2128a250c8376d6015caa94ec346fa5f1bf86d827a84025"}
Sep 17 15:40:57.415451 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.415381    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:57.416167 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.416143    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:57.647330 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.647290    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:40:57.977497 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:57.977469    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:40:58.105266 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.105231    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b5xqc"
Sep 17 15:40:58.244355 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.244282    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd" event={"ID":"ffa61525-735b-40bd-89ea-ccf07b290ab7","Type":"ContainerStarted","Data":"5378901aab92400967788451db2d4e5b7e7f75125753b2368e2fff62eb8fbf2a"}
Sep 17 15:40:58.246508 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.246476    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdf8" event={"ID":"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf","Type":"ContainerStarted","Data":"de3d937a08747af695726de85e003caa59a7dc37ab4b49f7c278c9882379a30e"}
Sep 17 15:40:58.246620 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.246524    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwdf8" event={"ID":"8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf","Type":"ContainerStarted","Data":"5eb08052e616eea620323d27bc15c3a505338a4e7a8795decf1d9ac45fa939c1"}
Sep 17 15:40:58.271901 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:58.271870    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4656a07f_62a5_4600_ba7e_f2e9d1e1d0c2.slice/crio-32808559f12302f0112eaada7a39f4f05f664d8f4f97519674286b0c16da8066 WatchSource:0}: Error finding container 32808559f12302f0112eaada7a39f4f05f664d8f4f97519674286b0c16da8066: Status 404 returned error can't find the container with id 32808559f12302f0112eaada7a39f4f05f664d8f4f97519674286b0c16da8066
Sep 17 15:40:58.280007 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.279961    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-hcvxd" podStartSLOduration=2.185959966 podStartE2EDuration="3.279946927s" totalImagesPullingTime="1.093986961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:58.27927431 +0000 UTC m=+102.867318896" watchObservedRunningTime="2026-09-17 15:40:58.279946927 +0000 UTC m=+102.867991513"
Sep 17 15:40:58.315223 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.315175    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pwdf8" podStartSLOduration=2.343636682 podStartE2EDuration="3.315159154s" totalImagesPullingTime="902.790931ms" totalInitContainerRuntime="68.731541ms" isStatefulPod=true podCreationTimestamp="2026-09-17 15:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:58.312992381 +0000 UTC m=+102.901036969" watchObservedRunningTime="2026-09-17 15:40:58.315159154 +0000 UTC m=+102.903203741"
Sep 17 15:40:58.760539 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.760448    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f75895595-jkq8h"]
Sep 17 15:40:58.787871 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.787827    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f75895595-jkq8h"]
Sep 17 15:40:58.787991 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.787946    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.791392 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.791199    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 17 15:40:58.792579 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.791630    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 17 15:40:58.792579 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.791741    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 17 15:40:58.792579 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.791911    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 17 15:40:58.792579 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.792239    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tp29q\""
Sep 17 15:40:58.792579 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.792299    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 17 15:40:58.792579 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.792388    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-da3871q19oocv\""
Sep 17 15:40:58.931627 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931572    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.931627 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931617    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-grpc-tls\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.931885 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931646    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.931885 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931720    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5469f1f4-6bfb-48fc-a2cf-a606625648df-metrics-client-ca\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.931885 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.931885 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931780    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2f8q\" (UniqueName: \"kubernetes.io/projected/5469f1f4-6bfb-48fc-a2cf-a606625648df-kube-api-access-c2f8q\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.932056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931879    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-tls\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:58.932056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:58.931938    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.032922 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.032812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.032922 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.032900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-grpc-tls\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.033429 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.032935    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.033429 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.032971    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5469f1f4-6bfb-48fc-a2cf-a606625648df-metrics-client-ca\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.033429 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.033003    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.033429 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.033034    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2f8q\" (UniqueName: \"kubernetes.io/projected/5469f1f4-6bfb-48fc-a2cf-a606625648df-kube-api-access-c2f8q\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.033429 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.033081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-tls\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.033429 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.033117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.033822 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.033769    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5469f1f4-6bfb-48fc-a2cf-a606625648df-metrics-client-ca\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.035496 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.035464    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.035766 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.035734    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.035882 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.035828    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.036045 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.036026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.036218 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.036201    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-grpc-tls\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.036271 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.036208    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5469f1f4-6bfb-48fc-a2cf-a606625648df-secret-thanos-querier-tls\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.043623 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.043599    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2f8q\" (UniqueName: \"kubernetes.io/projected/5469f1f4-6bfb-48fc-a2cf-a606625648df-kube-api-access-c2f8q\") pod \"thanos-querier-7f75895595-jkq8h\" (UID: \"5469f1f4-6bfb-48fc-a2cf-a606625648df\") " pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.101097 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.101063    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:40:59.253978 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.253934    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb" event={"ID":"b870e596-5e4e-41ba-9c45-d694ab3d50a5","Type":"ContainerStarted","Data":"464c831e2f47ce893d2423848eefb595ebc7722838ed3613919bcbfe70ba2b40"}
Sep 17 15:40:59.254155 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.254013    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb" event={"ID":"b870e596-5e4e-41ba-9c45-d694ab3d50a5","Type":"ContainerStarted","Data":"166ce5998753bc39ebb7975497e36a0d380403d2e01577183ef7d4a0d23a9b10"}
Sep 17 15:40:59.254155 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.254042    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb" event={"ID":"b870e596-5e4e-41ba-9c45-d694ab3d50a5","Type":"ContainerStarted","Data":"b0798424f0caeadf13aa36b0945782b0f24a778e466b59dee568688d98fbe340"}
Sep 17 15:40:59.255404 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.255375    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerStarted","Data":"32808559f12302f0112eaada7a39f4f05f664d8f4f97519674286b0c16da8066"}
Sep 17 15:40:59.279704 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.279648    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-gkmrb" podStartSLOduration=2.647831391 podStartE2EDuration="4.279628237s" totalImagesPullingTime="1.631796846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:40:59.278919731 +0000 UTC m=+103.866964320" watchObservedRunningTime="2026-09-17 15:40:59.279628237 +0000 UTC m=+103.867672823"
Sep 17 15:40:59.653466 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:40:59.651694    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f75895595-jkq8h"]
Sep 17 15:40:59.697470 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:40:59.697436    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5469f1f4_6bfb_48fc_a2cf_a606625648df.slice/crio-6ee4b8fc20ae33da74cf8c6e4915e94ac15eb366cfd10cde9b594fcfca289ef4 WatchSource:0}: Error finding container 6ee4b8fc20ae33da74cf8c6e4915e94ac15eb366cfd10cde9b594fcfca289ef4: Status 404 returned error can't find the container with id 6ee4b8fc20ae33da74cf8c6e4915e94ac15eb366cfd10cde9b594fcfca289ef4
Sep 17 15:41:00.006244 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.006207    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"]
Sep 17 15:41:00.020562 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.020534    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.024691 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.024663    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 17 15:41:00.024834 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.024691    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 17 15:41:00.024834 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.024669    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ts7kx\""
Sep 17 15:41:00.025214 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.025196    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 17 15:41:00.025431 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.025409    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-a62inmnhohs5t\""
Sep 17 15:41:00.025671 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.025651    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 15:41:00.027391 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.027252    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"]
Sep 17 15:41:00.142637 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.142601    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-secret-metrics-server-tls\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.142637 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.142639    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6135a1b1-7a60-4d25-aecc-92a183de524f-metrics-server-audit-profiles\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.143177 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.142709    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6135a1b1-7a60-4d25-aecc-92a183de524f-audit-log\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.143177 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.142763    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-secret-metrics-server-client-certs\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.143177 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.142813    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6135a1b1-7a60-4d25-aecc-92a183de524f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.143177 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.142868    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f7cj\" (UniqueName: \"kubernetes.io/projected/6135a1b1-7a60-4d25-aecc-92a183de524f-kube-api-access-6f7cj\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.143177 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.142933    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-client-ca-bundle\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.244338 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.244303    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-client-ca-bundle\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.244528 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.244378    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-secret-metrics-server-tls\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.244528 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.244411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6135a1b1-7a60-4d25-aecc-92a183de524f-metrics-server-audit-profiles\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.244528 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.244463    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6135a1b1-7a60-4d25-aecc-92a183de524f-audit-log\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.244528 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.244498    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-secret-metrics-server-client-certs\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.244732 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.244535    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6135a1b1-7a60-4d25-aecc-92a183de524f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.244732 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.244560    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f7cj\" (UniqueName: \"kubernetes.io/projected/6135a1b1-7a60-4d25-aecc-92a183de524f-kube-api-access-6f7cj\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.245056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.245007    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6135a1b1-7a60-4d25-aecc-92a183de524f-audit-log\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.245559 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.245521    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6135a1b1-7a60-4d25-aecc-92a183de524f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.245680 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.245592    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6135a1b1-7a60-4d25-aecc-92a183de524f-metrics-server-audit-profiles\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.247353 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.247328    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-secret-metrics-server-tls\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.247464 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.247378    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-client-ca-bundle\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.247464 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.247449    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6135a1b1-7a60-4d25-aecc-92a183de524f-secret-metrics-server-client-certs\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.260784 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.260721    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f7cj\" (UniqueName: \"kubernetes.io/projected/6135a1b1-7a60-4d25-aecc-92a183de524f-kube-api-access-6f7cj\") pod \"metrics-server-85b6bfbb6d-wtwrn\" (UID: \"6135a1b1-7a60-4d25-aecc-92a183de524f\") " pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.262548 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.262516    2568 generic.go:356] "Generic (PLEG): container finished" podID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerID="7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca" exitCode=0
Sep 17 15:41:00.262668 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.262608    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca"}
Sep 17 15:41:00.264223 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.264201    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" event={"ID":"5469f1f4-6bfb-48fc-a2cf-a606625648df","Type":"ContainerStarted","Data":"6ee4b8fc20ae33da74cf8c6e4915e94ac15eb366cfd10cde9b594fcfca289ef4"}
Sep 17 15:41:00.330554 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.330514    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:00.645135 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:00.645107    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"]
Sep 17 15:41:00.645463 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:41:00.645433    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6135a1b1_7a60_4d25_aecc_92a183de524f.slice/crio-23a8c21236fc3fc8e84c748eb4a13e57599edefc4513feca82d9bf1a81aa11ca WatchSource:0}: Error finding container 23a8c21236fc3fc8e84c748eb4a13e57599edefc4513feca82d9bf1a81aa11ca: Status 404 returned error can't find the container with id 23a8c21236fc3fc8e84c748eb4a13e57599edefc4513feca82d9bf1a81aa11ca
Sep 17 15:41:01.270143 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:01.270104    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn" event={"ID":"6135a1b1-7a60-4d25-aecc-92a183de524f","Type":"ContainerStarted","Data":"23a8c21236fc3fc8e84c748eb4a13e57599edefc4513feca82d9bf1a81aa11ca"}
Sep 17 15:41:02.278065 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:02.278027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerStarted","Data":"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"}
Sep 17 15:41:02.278483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:02.278142    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerStarted","Data":"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"}
Sep 17 15:41:02.278483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:02.278189    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerStarted","Data":"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"}
Sep 17 15:41:02.278483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:02.278229    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerStarted","Data":"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"}
Sep 17 15:41:02.280499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:02.280468    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" event={"ID":"5469f1f4-6bfb-48fc-a2cf-a606625648df","Type":"ContainerStarted","Data":"1c19692680ce6daf61c5a560c300ca11422947126601dc89e7e8042fccf0144d"}
Sep 17 15:41:02.280605 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:02.280545    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" event={"ID":"5469f1f4-6bfb-48fc-a2cf-a606625648df","Type":"ContainerStarted","Data":"02e7275378983db281edac248e16afacf5547f12d01f705bccb75b85c3daf6c2"}
Sep 17 15:41:02.280605 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:02.280586    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" event={"ID":"5469f1f4-6bfb-48fc-a2cf-a606625648df","Type":"ContainerStarted","Data":"bf995cbf6bed29be414cab5be3533d30a7382fc3d07f7868fb0deb431f612052"}
Sep 17 15:41:03.287777 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.287676    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerStarted","Data":"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"}
Sep 17 15:41:03.287777 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.287757    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerStarted","Data":"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"}
Sep 17 15:41:03.289345 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.289315    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn" event={"ID":"6135a1b1-7a60-4d25-aecc-92a183de524f","Type":"ContainerStarted","Data":"ca57e59fd2357f2d8f97ccffe7e277e847946dc1e141116a80f0612131e1c8b0"}
Sep 17 15:41:03.292241 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.292218    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" event={"ID":"5469f1f4-6bfb-48fc-a2cf-a606625648df","Type":"ContainerStarted","Data":"806e5e1b847b87191e9c91ebd0433ba2e2968b574269b2121a4e08af6b393d5c"}
Sep 17 15:41:03.292333 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.292274    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" event={"ID":"5469f1f4-6bfb-48fc-a2cf-a606625648df","Type":"ContainerStarted","Data":"a2249e10f74b2c14657754e7802bda791372a600f17dae211ae5beb22296489e"}
Sep 17 15:41:03.292333 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.292302    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" event={"ID":"5469f1f4-6bfb-48fc-a2cf-a606625648df","Type":"ContainerStarted","Data":"0885e983a8d444d11ee083708a68b585dc91fa764da6384ea3f1e698407d6ff2"}
Sep 17 15:41:03.292447 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.292429    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:41:03.338932 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.338887    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.793939776 podStartE2EDuration="7.338872342s" totalImagesPullingTime="3.532104336s" totalInitContainerRuntime="12.82823ms" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:41:03.336480174 +0000 UTC m=+107.924524763" watchObservedRunningTime="2026-09-17 15:41:03.338872342 +0000 UTC m=+107.926916929"
Sep 17 15:41:03.362883 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.362811    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn" podStartSLOduration=2.4357016 podStartE2EDuration="4.362795536s" totalImagesPullingTime="1.927093936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:41:03.361394665 +0000 UTC m=+107.949439252" watchObservedRunningTime="2026-09-17 15:41:03.362795536 +0000 UTC m=+107.950840123"
Sep 17 15:41:03.408139 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:03.408093    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h" podStartSLOduration=2.313858686 podStartE2EDuration="5.408076585s" totalImagesPullingTime="3.094217899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:41:03.405793085 +0000 UTC m=+107.993837672" watchObservedRunningTime="2026-09-17 15:41:03.408076585 +0000 UTC m=+107.996121171"
Sep 17 15:41:05.181771 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:05.181740    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6df7765dc8-85gxv"
Sep 17 15:41:06.239394 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.239360    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-llms2"
Sep 17 15:41:06.525953 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.525858    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q" podUID="82fc6885-94c5-4d96-888a-a71e58344ebb" containerName="registry" containerID="cri-o://1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31" gracePeriod=30
Sep 17 15:41:06.765270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.765247    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:41:06.908858 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.908759    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82fc6885-94c5-4d96-888a-a71e58344ebb-ca-trust-extracted\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.908858 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.908801    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-image-registry-private-configuration\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.908858 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.908833    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.909133 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.908879    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-trusted-ca\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.909133 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.908930    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-certificates\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.909133 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.908965    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-bound-sa-token\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.909133 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.909049    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-installation-pull-secrets\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.909133 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.909081    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prlp5\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-kube-api-access-prlp5\") pod \"82fc6885-94c5-4d96-888a-a71e58344ebb\" (UID: \"82fc6885-94c5-4d96-888a-a71e58344ebb\") "
Sep 17 15:41:06.909503 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.909468    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:41:06.909638 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.909484    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:41:06.911395 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.911368    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:41:06.911601 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.911457    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-kube-api-access-prlp5" (OuterVolumeSpecName: "kube-api-access-prlp5") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "kube-api-access-prlp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:41:06.911601 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.911592    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:41:06.912065 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.912024    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:41:06.912559 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.912539    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:41:06.918468 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:06.918444    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82fc6885-94c5-4d96-888a-a71e58344ebb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "82fc6885-94c5-4d96-888a-a71e58344ebb" (UID: "82fc6885-94c5-4d96-888a-a71e58344ebb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 15:41:07.009798 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009761    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82fc6885-94c5-4d96-888a-a71e58344ebb-ca-trust-extracted\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.009798 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009792    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-image-registry-private-configuration\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.009798 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009802    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.010056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009812    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-trusted-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.010056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009820    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82fc6885-94c5-4d96-888a-a71e58344ebb-registry-certificates\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.010056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009828    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-bound-sa-token\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.010056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009860    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82fc6885-94c5-4d96-888a-a71e58344ebb-installation-pull-secrets\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.010056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.009869    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prlp5\" (UniqueName: \"kubernetes.io/projected/82fc6885-94c5-4d96-888a-a71e58344ebb-kube-api-access-prlp5\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:41:07.313784 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.313751    2568 generic.go:356] "Generic (PLEG): container finished" podID="82fc6885-94c5-4d96-888a-a71e58344ebb" containerID="1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31" exitCode=0
Sep 17 15:41:07.314219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.313800    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q"
Sep 17 15:41:07.314219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.313830    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q" event={"ID":"82fc6885-94c5-4d96-888a-a71e58344ebb","Type":"ContainerDied","Data":"1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31"}
Sep 17 15:41:07.314219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.313889    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f69d7c84-59v6q" event={"ID":"82fc6885-94c5-4d96-888a-a71e58344ebb","Type":"ContainerDied","Data":"a19b99c9c955d3ecac34d43aa1ea25f5eccb60d41599111d3d51e3870f86afa6"}
Sep 17 15:41:07.314219 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.313910    2568 scope.go:117] "RemoveContainer" containerID="1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31"
Sep 17 15:41:07.321938 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.321919    2568 scope.go:117] "RemoveContainer" containerID="1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31"
Sep 17 15:41:07.322244 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:41:07.322223    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31\": container with ID starting with 1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31 not found: ID does not exist" containerID="1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31"
Sep 17 15:41:07.322284 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.322252    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31"} err="failed to get container status \"1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31\": rpc error: code = NotFound desc = could not find container \"1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31\": container with ID starting with 1f51d58d4273652d342aa49bad364da877d7571c55b89fe4967f2d591a31fb31 not found: ID does not exist"
Sep 17 15:41:07.337117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.337093    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8f69d7c84-59v6q"]
Sep 17 15:41:07.344265 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.344244    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8f69d7c84-59v6q"]
Sep 17 15:41:07.835024 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:07.834990    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82fc6885-94c5-4d96-888a-a71e58344ebb" path="/var/lib/kubelet/pods/82fc6885-94c5-4d96-888a-a71e58344ebb/volumes"
Sep 17 15:41:09.306620 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:09.306592    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f75895595-jkq8h"
Sep 17 15:41:10.826185 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:10.826157    2568 scope.go:117] "RemoveContainer" containerID="98ff177dcd62bb757f74a7c944b2f042b1d192fc2e90795998830e01ac10a84b"
Sep 17 15:41:11.332322 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:11.332290    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:41:11.332502 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:11.332409    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" event={"ID":"8e244896-d0b1-4caf-ae7b-a31ea72fad9c","Type":"ContainerStarted","Data":"f1fd13f55396934644cd18f319172808548156fb5beb812a57e61629244c8245"}
Sep 17 15:41:11.332672 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:11.332658    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:41:11.337618 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:11.337593    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b"
Sep 17 15:41:11.356704 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:11.356660    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-d8g8b" podStartSLOduration=56.214272858 podStartE2EDuration="58.356646828s" totalImagesPullingTime="2.14237397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:41:11.35619732 +0000 UTC m=+115.944241907" watchObservedRunningTime="2026-09-17 15:41:11.356646828 +0000 UTC m=+115.944691465"
Sep 17 15:41:18.227230 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.227196    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6499c9bc95-76mfz"]
Sep 17 15:41:18.227589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.227489    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82fc6885-94c5-4d96-888a-a71e58344ebb" containerName="registry"
Sep 17 15:41:18.227589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.227500    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="82fc6885-94c5-4d96-888a-a71e58344ebb" containerName="registry"
Sep 17 15:41:18.227589 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.227558    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="82fc6885-94c5-4d96-888a-a71e58344ebb" containerName="registry"
Sep 17 15:41:18.285532 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.285502    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6499c9bc95-76mfz"]
Sep 17 15:41:18.285678 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.285631    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.288562 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.288535    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 15:41:18.288682 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.288619    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 15:41:18.288860 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.288830    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 15:41:18.288860 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.288852    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 15:41:18.288995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.288852    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 15:41:18.288995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.288910    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 15:41:18.288995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.288982    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w694p\""
Sep 17 15:41:18.289143 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.289017    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 15:41:18.408331 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.408301    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cmx5\" (UniqueName: \"kubernetes.io/projected/a6b91299-7347-4fea-9ac9-5fb6275def2d-kube-api-access-9cmx5\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.408490 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.408346    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-service-ca\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.408490 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.408417    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-config\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.408490 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.408448    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-serving-cert\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.408490 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.408469    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-oauth-serving-cert\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.408648 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.408521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-oauth-config\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.509717 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.509641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-service-ca\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.509717 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.509702    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-config\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.509924 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.509731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-serving-cert\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.509924 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.509756    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-oauth-serving-cert\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.510016 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.509921    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-oauth-config\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.510016 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.509968    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cmx5\" (UniqueName: \"kubernetes.io/projected/a6b91299-7347-4fea-9ac9-5fb6275def2d-kube-api-access-9cmx5\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.510475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.510446    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-config\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.510475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.510464    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-oauth-serving-cert\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.510662 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.510464    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-service-ca\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.512233 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.512213    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-oauth-config\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.512414 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.512395    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-serving-cert\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.524343 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.524317    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cmx5\" (UniqueName: \"kubernetes.io/projected/a6b91299-7347-4fea-9ac9-5fb6275def2d-kube-api-access-9cmx5\") pod \"console-6499c9bc95-76mfz\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") " pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.595337 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.595307    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:18.922147 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:18.922074    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6499c9bc95-76mfz"]
Sep 17 15:41:18.924945 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:41:18.924920    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b91299_7347_4fea_9ac9_5fb6275def2d.slice/crio-cc6d958e8380cb7bdc149114dbc2ebb9cd2784e33fd39110964f732227b0795a WatchSource:0}: Error finding container cc6d958e8380cb7bdc149114dbc2ebb9cd2784e33fd39110964f732227b0795a: Status 404 returned error can't find the container with id cc6d958e8380cb7bdc149114dbc2ebb9cd2784e33fd39110964f732227b0795a
Sep 17 15:41:19.369530 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:19.369484    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c9bc95-76mfz" event={"ID":"a6b91299-7347-4fea-9ac9-5fb6275def2d","Type":"ContainerStarted","Data":"cc6d958e8380cb7bdc149114dbc2ebb9cd2784e33fd39110964f732227b0795a"}
Sep 17 15:41:20.330711 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:20.330674    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:20.331025 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:20.330740    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:22.382819 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:22.382785    2568 generic.go:356] "Generic (PLEG): container finished" podID="f5b24329-b842-4a2e-b517-ae2adea120e5" containerID="d014684026a19f3d463de1936028f91f30d1a7f60a302de52f2d877247ff5388" exitCode=0
Sep 17 15:41:22.383236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:22.382874    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf" event={"ID":"f5b24329-b842-4a2e-b517-ae2adea120e5","Type":"ContainerDied","Data":"d014684026a19f3d463de1936028f91f30d1a7f60a302de52f2d877247ff5388"}
Sep 17 15:41:22.383293 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:22.383234    2568 scope.go:117] "RemoveContainer" containerID="d014684026a19f3d463de1936028f91f30d1a7f60a302de52f2d877247ff5388"
Sep 17 15:41:22.384659 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:22.384636    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c9bc95-76mfz" event={"ID":"a6b91299-7347-4fea-9ac9-5fb6275def2d","Type":"ContainerStarted","Data":"cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47"}
Sep 17 15:41:22.446548 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:22.446498    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6499c9bc95-76mfz" podStartSLOduration=1.887710617 podStartE2EDuration="4.446473789s" totalImagesPullingTime="2.558763172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:41:22.445626026 +0000 UTC m=+127.033670614" watchObservedRunningTime="2026-09-17 15:41:22.446473789 +0000 UTC m=+127.034518378"
Sep 17 15:41:23.390206 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:23.390166    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-26xgf" event={"ID":"f5b24329-b842-4a2e-b517-ae2adea120e5","Type":"ContainerStarted","Data":"e2f9f9751e82084b2cbdab2527e21e82ae821a8b171f729977f4fdf4af3ff6e5"}
Sep 17 15:41:25.578405 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:25.578363    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:41:25.580657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:25.580631    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ff164ba-34fa-45aa-b854-e48384d80808-metrics-certs\") pod \"network-metrics-daemon-cgmvm\" (UID: \"7ff164ba-34fa-45aa-b854-e48384d80808\") " pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:41:25.640931 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:25.640906    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ddz86\""
Sep 17 15:41:25.648998 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:25.648980    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgmvm"
Sep 17 15:41:25.921587 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:25.921552    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cgmvm"]
Sep 17 15:41:25.923333 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:41:25.923301    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff164ba_34fa_45aa_b854_e48384d80808.slice/crio-ad351c67a7fe39f8bf664f157ba198f3663ed4a49b304fd8cbc1f88925128dd9 WatchSource:0}: Error finding container ad351c67a7fe39f8bf664f157ba198f3663ed4a49b304fd8cbc1f88925128dd9: Status 404 returned error can't find the container with id ad351c67a7fe39f8bf664f157ba198f3663ed4a49b304fd8cbc1f88925128dd9
Sep 17 15:41:26.404601 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:26.404565    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgmvm" event={"ID":"7ff164ba-34fa-45aa-b854-e48384d80808","Type":"ContainerStarted","Data":"ad351c67a7fe39f8bf664f157ba198f3663ed4a49b304fd8cbc1f88925128dd9"}
Sep 17 15:41:27.337555 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.337520    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c7d8dc7b7-rqqvs"]
Sep 17 15:41:27.339784 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.339769    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.348727 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.348703    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 15:41:27.356925 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.356895    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7d8dc7b7-rqqvs"]
Sep 17 15:41:27.396939 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.396914    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-oauth-config\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.397060 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.396960    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-trusted-ca-bundle\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.397060 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.396979    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lqxc\" (UniqueName: \"kubernetes.io/projected/0e033ad9-34f4-4029-9d01-03252473bdcc-kube-api-access-4lqxc\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.397060 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.397003    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-serving-cert\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.397060 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.397041    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-console-config\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.397060 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.397058    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-service-ca\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.397274 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.397133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-oauth-serving-cert\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.409773 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.409749    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgmvm" event={"ID":"7ff164ba-34fa-45aa-b854-e48384d80808","Type":"ContainerStarted","Data":"ed801ac1e7a259ba895b0421a0aa1e9648ac9e62e30e584b73ec82a1ee2ea94f"}
Sep 17 15:41:27.409877 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.409795    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgmvm" event={"ID":"7ff164ba-34fa-45aa-b854-e48384d80808","Type":"ContainerStarted","Data":"fd00c183b03940f11493248201c39ad137fc90590b169a733720efcaf1abc601"}
Sep 17 15:41:27.436599 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.436552    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cgmvm" podStartSLOduration=131.598977861 podStartE2EDuration="2m12.436538408s" totalImagesPullingTime="837.560547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:41:27.436082263 +0000 UTC m=+132.024126851" watchObservedRunningTime="2026-09-17 15:41:27.436538408 +0000 UTC m=+132.024582994"
Sep 17 15:41:27.498212 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.498176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-oauth-config\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.498364 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.498232    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-trusted-ca-bundle\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.498444 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.498360    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lqxc\" (UniqueName: \"kubernetes.io/projected/0e033ad9-34f4-4029-9d01-03252473bdcc-kube-api-access-4lqxc\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.498444 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.498395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-serving-cert\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.498542 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.498441    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-console-config\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.498595 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.498571    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-service-ca\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.498648 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.498630    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-oauth-serving-cert\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.499205 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.499177    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-trusted-ca-bundle\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.499320 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.499275    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-service-ca\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.499397 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.499375    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-oauth-serving-cert\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.499706 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.499686    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-console-config\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.500952 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.500932    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-serving-cert\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.501032 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.500955    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-oauth-config\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.508514 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.508486    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lqxc\" (UniqueName: \"kubernetes.io/projected/0e033ad9-34f4-4029-9d01-03252473bdcc-kube-api-access-4lqxc\") pod \"console-7c7d8dc7b7-rqqvs\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") " pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.648866 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.648769    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:27.923156 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:27.923078    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7d8dc7b7-rqqvs"]
Sep 17 15:41:27.926247 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:41:27.926221    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e033ad9_34f4_4029_9d01_03252473bdcc.slice/crio-44054f147ab29ed6a37736c2cdc48c84db4f7607e8ab1c1bb5481f0f63a37566 WatchSource:0}: Error finding container 44054f147ab29ed6a37736c2cdc48c84db4f7607e8ab1c1bb5481f0f63a37566: Status 404 returned error can't find the container with id 44054f147ab29ed6a37736c2cdc48c84db4f7607e8ab1c1bb5481f0f63a37566
Sep 17 15:41:28.415026 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:28.414984    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7d8dc7b7-rqqvs" event={"ID":"0e033ad9-34f4-4029-9d01-03252473bdcc","Type":"ContainerStarted","Data":"5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34"}
Sep 17 15:41:28.415395 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:28.415042    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7d8dc7b7-rqqvs" event={"ID":"0e033ad9-34f4-4029-9d01-03252473bdcc","Type":"ContainerStarted","Data":"44054f147ab29ed6a37736c2cdc48c84db4f7607e8ab1c1bb5481f0f63a37566"}
Sep 17 15:41:28.441092 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:28.441046    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c7d8dc7b7-rqqvs" podStartSLOduration=1.441029875 podStartE2EDuration="1.441029875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:41:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:41:28.43750307 +0000 UTC m=+133.025547657" watchObservedRunningTime="2026-09-17 15:41:28.441029875 +0000 UTC m=+133.029074520"
Sep 17 15:41:28.595913 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:28.595879    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:28.596101 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:28.595948    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:28.600660 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:28.600636    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:29.422998 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:29.422969    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:41:29.992745 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:29.992712    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vsd8x_76cfc63e-e046-4228-aec2-8b884e290052/serve-healthcheck-canary/0.log"
Sep 17 15:41:37.649459 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:37.649412    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:37.650078 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:37.649494    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:37.654196 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:37.654173    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:38.461292 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:38.461259    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:41:38.563549 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:38.563514    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6499c9bc95-76mfz"]
Sep 17 15:41:40.338050 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:40.338017    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:41:40.342000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:41:40.341977    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85b6bfbb6d-wtwrn"
Sep 17 15:42:03.584380 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.584334    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6499c9bc95-76mfz" podUID="a6b91299-7347-4fea-9ac9-5fb6275def2d" containerName="console" containerID="cri-o://cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47" gracePeriod=15
Sep 17 15:42:03.829819 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.829788    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:42:03.899905 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.899815    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-oauth-serving-cert\") pod \"a6b91299-7347-4fea-9ac9-5fb6275def2d\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") "
Sep 17 15:42:03.899905 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.899873    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-serving-cert\") pod \"a6b91299-7347-4fea-9ac9-5fb6275def2d\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") "
Sep 17 15:42:03.899905 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.899901    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-oauth-config\") pod \"a6b91299-7347-4fea-9ac9-5fb6275def2d\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") "
Sep 17 15:42:03.900104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.899926    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-service-ca\") pod \"a6b91299-7347-4fea-9ac9-5fb6275def2d\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") "
Sep 17 15:42:03.900104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.899975    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-config\") pod \"a6b91299-7347-4fea-9ac9-5fb6275def2d\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") "
Sep 17 15:42:03.900104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.900014    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cmx5\" (UniqueName: \"kubernetes.io/projected/a6b91299-7347-4fea-9ac9-5fb6275def2d-kube-api-access-9cmx5\") pod \"a6b91299-7347-4fea-9ac9-5fb6275def2d\" (UID: \"a6b91299-7347-4fea-9ac9-5fb6275def2d\") "
Sep 17 15:42:03.900297 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.900274    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a6b91299-7347-4fea-9ac9-5fb6275def2d" (UID: "a6b91299-7347-4fea-9ac9-5fb6275def2d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:42:03.900362 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.900333    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-service-ca" (OuterVolumeSpecName: "service-ca") pod "a6b91299-7347-4fea-9ac9-5fb6275def2d" (UID: "a6b91299-7347-4fea-9ac9-5fb6275def2d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:42:03.900417 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.900376    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-config" (OuterVolumeSpecName: "console-config") pod "a6b91299-7347-4fea-9ac9-5fb6275def2d" (UID: "a6b91299-7347-4fea-9ac9-5fb6275def2d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:42:03.902221 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.902192    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a6b91299-7347-4fea-9ac9-5fb6275def2d" (UID: "a6b91299-7347-4fea-9ac9-5fb6275def2d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:03.902221 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.902202    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b91299-7347-4fea-9ac9-5fb6275def2d-kube-api-access-9cmx5" (OuterVolumeSpecName: "kube-api-access-9cmx5") pod "a6b91299-7347-4fea-9ac9-5fb6275def2d" (UID: "a6b91299-7347-4fea-9ac9-5fb6275def2d"). InnerVolumeSpecName "kube-api-access-9cmx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:42:03.902350 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:03.902232    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a6b91299-7347-4fea-9ac9-5fb6275def2d" (UID: "a6b91299-7347-4fea-9ac9-5fb6275def2d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:04.000783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.000745    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-oauth-serving-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:04.000783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.000778    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-serving-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:04.000783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.000787    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-oauth-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:04.000986 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.000797    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:04.000986 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.000807    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b91299-7347-4fea-9ac9-5fb6275def2d-console-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:04.000986 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.000814    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cmx5\" (UniqueName: \"kubernetes.io/projected/a6b91299-7347-4fea-9ac9-5fb6275def2d-kube-api-access-9cmx5\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:04.561793 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.561759    2568 generic.go:356] "Generic (PLEG): container finished" podID="a6b91299-7347-4fea-9ac9-5fb6275def2d" containerID="cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47" exitCode=0
Sep 17 15:42:04.561971 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.561815    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6499c9bc95-76mfz"
Sep 17 15:42:04.561971 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.561828    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c9bc95-76mfz" event={"ID":"a6b91299-7347-4fea-9ac9-5fb6275def2d","Type":"ContainerDied","Data":"cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47"}
Sep 17 15:42:04.561971 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.561924    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6499c9bc95-76mfz" event={"ID":"a6b91299-7347-4fea-9ac9-5fb6275def2d","Type":"ContainerDied","Data":"cc6d958e8380cb7bdc149114dbc2ebb9cd2784e33fd39110964f732227b0795a"}
Sep 17 15:42:04.561971 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.561946    2568 scope.go:117] "RemoveContainer" containerID="cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47"
Sep 17 15:42:04.572544 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.572529    2568 scope.go:117] "RemoveContainer" containerID="cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47"
Sep 17 15:42:04.572803 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:04.572783    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47\": container with ID starting with cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47 not found: ID does not exist" containerID="cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47"
Sep 17 15:42:04.572877 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.572812    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47"} err="failed to get container status \"cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47\": rpc error: code = NotFound desc = could not find container \"cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47\": container with ID starting with cc764dddd2e787c676885662adeb7468580226e2fbf63f5a8863ecd7efb3ea47 not found: ID does not exist"
Sep 17 15:42:04.592471 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.592443    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6499c9bc95-76mfz"]
Sep 17 15:42:04.597911 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:04.597884    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6499c9bc95-76mfz"]
Sep 17 15:42:05.831163 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:05.831125    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6b91299-7347-4fea-9ac9-5fb6275def2d" path="/var/lib/kubelet/pods/a6b91299-7347-4fea-9ac9-5fb6275def2d/volumes"
Sep 17 15:42:16.131400 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.131315    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:42:16.131888 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.131587    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="alertmanager" containerID="cri-o://f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be" gracePeriod=120
Sep 17 15:42:16.131888 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.131657    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-web" containerID="cri-o://13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550" gracePeriod=120
Sep 17 15:42:16.131888 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.131657    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-metric" containerID="cri-o://203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f" gracePeriod=120
Sep 17 15:42:16.131888 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.131704    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy" containerID="cri-o://17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153" gracePeriod=120
Sep 17 15:42:16.131888 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.131681    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="config-reloader" containerID="cri-o://f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1" gracePeriod=120
Sep 17 15:42:16.131888 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.131771    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="prom-label-proxy" containerID="cri-o://ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d" gracePeriod=120
Sep 17 15:42:16.615299 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615269    2568 generic.go:356] "Generic (PLEG): container finished" podID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerID="ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d" exitCode=0
Sep 17 15:42:16.615299 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615294    2568 generic.go:356] "Generic (PLEG): container finished" podID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerID="17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153" exitCode=0
Sep 17 15:42:16.615299 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615300    2568 generic.go:356] "Generic (PLEG): container finished" podID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerID="f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1" exitCode=0
Sep 17 15:42:16.615299 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615306    2568 generic.go:356] "Generic (PLEG): container finished" podID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerID="f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be" exitCode=0
Sep 17 15:42:16.615616 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615335    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"}
Sep 17 15:42:16.615616 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615397    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"}
Sep 17 15:42:16.615616 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615423    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"}
Sep 17 15:42:16.615616 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:16.615448    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"}
Sep 17 15:42:17.391528 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.391502    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.511483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511397    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-tls-assets\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511434    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511454    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5c6x\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-kube-api-access-t5c6x\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511483 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511480    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-metrics-client-ca\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511537    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-main-tls\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511572    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511657    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-cluster-tls-config\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511707    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511754    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.511830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511799    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-volume\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.512180 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511874    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-main-db\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.512180 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511912    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-out\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.512180 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511964    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-web-config\") pod \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\" (UID: \"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2\") "
Sep 17 15:42:17.512180 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.511965    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:42:17.512437 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.512228    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-metrics-client-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.514121 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.514054    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 15:42:17.514679 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.514637    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:17.514785 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.514681    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:42:17.515043 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.515010    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:42:17.515233 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.515203    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-kube-api-access-t5c6x" (OuterVolumeSpecName: "kube-api-access-t5c6x") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "kube-api-access-t5c6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:42:17.515416 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.515379    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-volume" (OuterVolumeSpecName: "config-volume") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:17.515594 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.515572    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:17.515749 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.515727    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:17.516380 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.516354    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:17.516380 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.516366    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-out" (OuterVolumeSpecName: "config-out") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 15:42:17.519738 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.519708    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:17.525994 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.525967    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-web-config" (OuterVolumeSpecName: "web-config") pod "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" (UID: "4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:42:17.613168 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613131    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-tls-assets\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613168 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613161    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613168 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613174    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5c6x\" (UniqueName: \"kubernetes.io/projected/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-kube-api-access-t5c6x\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613187    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-main-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613200    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613210    2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-cluster-tls-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613218    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613226    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613234    2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-volume\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613242    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-alertmanager-main-db\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613251    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-config-out\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.613393 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.613259    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2-web-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:42:17.623196 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.623163    2568 generic.go:356] "Generic (PLEG): container finished" podID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerID="203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f" exitCode=0
Sep 17 15:42:17.623196 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.623187    2568 generic.go:356] "Generic (PLEG): container finished" podID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerID="13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550" exitCode=0
Sep 17 15:42:17.623379 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.623238    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"}
Sep 17 15:42:17.623379 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.623262    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.623379 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.623311    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"}
Sep 17 15:42:17.623379 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.623339    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2","Type":"ContainerDied","Data":"32808559f12302f0112eaada7a39f4f05f664d8f4f97519674286b0c16da8066"}
Sep 17 15:42:17.623379 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.623378    2568 scope.go:117] "RemoveContainer" containerID="ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"
Sep 17 15:42:17.635046 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.634817    2568 scope.go:117] "RemoveContainer" containerID="203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"
Sep 17 15:42:17.642994 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.642970    2568 scope.go:117] "RemoveContainer" containerID="17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"
Sep 17 15:42:17.652118 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.651987    2568 scope.go:117] "RemoveContainer" containerID="13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"
Sep 17 15:42:17.654455 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.654430    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:42:17.660569 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.660542    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:42:17.661370 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.661354    2568 scope.go:117] "RemoveContainer" containerID="f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"
Sep 17 15:42:17.668634 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.668608    2568 scope.go:117] "RemoveContainer" containerID="f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"
Sep 17 15:42:17.675877 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.675855    2568 scope.go:117] "RemoveContainer" containerID="7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca"
Sep 17 15:42:17.683084 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.683062    2568 scope.go:117] "RemoveContainer" containerID="ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"
Sep 17 15:42:17.683368 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:17.683349    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d\": container with ID starting with ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d not found: ID does not exist" containerID="ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"
Sep 17 15:42:17.683433 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.683376    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"} err="failed to get container status \"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d\": rpc error: code = NotFound desc = could not find container \"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d\": container with ID starting with ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d not found: ID does not exist"
Sep 17 15:42:17.683433 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.683394    2568 scope.go:117] "RemoveContainer" containerID="203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"
Sep 17 15:42:17.683660 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:17.683641    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f\": container with ID starting with 203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f not found: ID does not exist" containerID="203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"
Sep 17 15:42:17.683700 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.683667    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"} err="failed to get container status \"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f\": rpc error: code = NotFound desc = could not find container \"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f\": container with ID starting with 203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f not found: ID does not exist"
Sep 17 15:42:17.683700 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.683682    2568 scope.go:117] "RemoveContainer" containerID="17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"
Sep 17 15:42:17.683956 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:17.683937    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153\": container with ID starting with 17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153 not found: ID does not exist" containerID="17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"
Sep 17 15:42:17.684006 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.683962    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"} err="failed to get container status \"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153\": rpc error: code = NotFound desc = could not find container \"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153\": container with ID starting with 17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153 not found: ID does not exist"
Sep 17 15:42:17.684006 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.683979    2568 scope.go:117] "RemoveContainer" containerID="13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"
Sep 17 15:42:17.684214 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:17.684199    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550\": container with ID starting with 13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550 not found: ID does not exist" containerID="13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"
Sep 17 15:42:17.684255 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684217    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"} err="failed to get container status \"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550\": rpc error: code = NotFound desc = could not find container \"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550\": container with ID starting with 13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550 not found: ID does not exist"
Sep 17 15:42:17.684255 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684230    2568 scope.go:117] "RemoveContainer" containerID="f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"
Sep 17 15:42:17.684453 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:17.684434    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1\": container with ID starting with f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1 not found: ID does not exist" containerID="f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"
Sep 17 15:42:17.684522 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684455    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"} err="failed to get container status \"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1\": rpc error: code = NotFound desc = could not find container \"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1\": container with ID starting with f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1 not found: ID does not exist"
Sep 17 15:42:17.684522 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684467    2568 scope.go:117] "RemoveContainer" containerID="f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"
Sep 17 15:42:17.684677 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:17.684662    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be\": container with ID starting with f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be not found: ID does not exist" containerID="f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"
Sep 17 15:42:17.684715 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684679    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"} err="failed to get container status \"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be\": rpc error: code = NotFound desc = could not find container \"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be\": container with ID starting with f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be not found: ID does not exist"
Sep 17 15:42:17.684715 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684695    2568 scope.go:117] "RemoveContainer" containerID="7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca"
Sep 17 15:42:17.684931 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:42:17.684915    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca\": container with ID starting with 7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca not found: ID does not exist" containerID="7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca"
Sep 17 15:42:17.684985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684932    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca"} err="failed to get container status \"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca\": rpc error: code = NotFound desc = could not find container \"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca\": container with ID starting with 7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca not found: ID does not exist"
Sep 17 15:42:17.684985 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.684943    2568 scope.go:117] "RemoveContainer" containerID="ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"
Sep 17 15:42:17.685190 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685171    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d"} err="failed to get container status \"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d\": rpc error: code = NotFound desc = could not find container \"ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d\": container with ID starting with ce0b88ed47fb115ea7c9256766647f250e18af131d8d2a6b1ac9ef88f552b20d not found: ID does not exist"
Sep 17 15:42:17.685237 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685191    2568 scope.go:117] "RemoveContainer" containerID="203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"
Sep 17 15:42:17.685408 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685391    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f"} err="failed to get container status \"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f\": rpc error: code = NotFound desc = could not find container \"203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f\": container with ID starting with 203b9ebb358ee4757a0eb524c394ed425b4f11a6ed2a84151714a3929bd79a8f not found: ID does not exist"
Sep 17 15:42:17.685408 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685406    2568 scope.go:117] "RemoveContainer" containerID="17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"
Sep 17 15:42:17.685648 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685630    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153"} err="failed to get container status \"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153\": rpc error: code = NotFound desc = could not find container \"17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153\": container with ID starting with 17123ec18a611087f8c135f56abb458c33ed7f8a5f47041a5cd5b9890cc52153 not found: ID does not exist"
Sep 17 15:42:17.685693 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685648    2568 scope.go:117] "RemoveContainer" containerID="13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"
Sep 17 15:42:17.685867 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685850    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550"} err="failed to get container status \"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550\": rpc error: code = NotFound desc = could not find container \"13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550\": container with ID starting with 13fed64f50b5ac3ef2f6d9b971cb5762a863b3e1c3410e3791cbe218ff313550 not found: ID does not exist"
Sep 17 15:42:17.685867 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.685866    2568 scope.go:117] "RemoveContainer" containerID="f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"
Sep 17 15:42:17.686091 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.686073    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1"} err="failed to get container status \"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1\": rpc error: code = NotFound desc = could not find container \"f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1\": container with ID starting with f1fedf5eff42e7b4169ef41f9834db6e21883b4b2bb3c82dd5ba2c212201d6c1 not found: ID does not exist"
Sep 17 15:42:17.686133 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.686091    2568 scope.go:117] "RemoveContainer" containerID="f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"
Sep 17 15:42:17.686298 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.686278    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be"} err="failed to get container status \"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be\": rpc error: code = NotFound desc = could not find container \"f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be\": container with ID starting with f8e269ae701c724b63f796e1042aa156e6d0cee1258824227c8a92cb0f2191be not found: ID does not exist"
Sep 17 15:42:17.686341 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.686299    2568 scope.go:117] "RemoveContainer" containerID="7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca"
Sep 17 15:42:17.686501 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.686485    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca"} err="failed to get container status \"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca\": rpc error: code = NotFound desc = could not find container \"7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca\": container with ID starting with 7acbb440816ac04d72bdb7e12c2bdc27f1cd4151d5c84abc43667cb585d689ca not found: ID does not exist"
Sep 17 15:42:17.689290 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689269    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:42:17.689615 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689599    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-web"
Sep 17 15:42:17.689657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689617    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-web"
Sep 17 15:42:17.689657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689625    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6b91299-7347-4fea-9ac9-5fb6275def2d" containerName="console"
Sep 17 15:42:17.689657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689630    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6b91299-7347-4fea-9ac9-5fb6275def2d" containerName="console"
Sep 17 15:42:17.689657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689640    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="prom-label-proxy"
Sep 17 15:42:17.689657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689646    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="prom-label-proxy"
Sep 17 15:42:17.689657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689659    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="alertmanager"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689664    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="alertmanager"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689670    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689675    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689681    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="config-reloader"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689686    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="config-reloader"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689691    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-metric"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689696    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-metric"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689701    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="init-config-reloader"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689706    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="init-config-reloader"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689749    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="alertmanager"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689758    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-web"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689763    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy-metric"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689769    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="config-reloader"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689774    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="prom-label-proxy"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689779    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6b91299-7347-4fea-9ac9-5fb6275def2d" containerName="console"
Sep 17 15:42:17.689827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.689787    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" containerName="kube-rbac-proxy"
Sep 17 15:42:17.694812 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.694794    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.697259 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697238    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 15:42:17.697373 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697273    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 15:42:17.697435 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697418    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 15:42:17.697513 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697498    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 15:42:17.697563 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697535    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 15:42:17.697639 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697622    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 15:42:17.697736 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697714    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 15:42:17.697827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697801    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 15:42:17.697941 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.697928    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h9nm4\""
Sep 17 15:42:17.702882 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.702865    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 15:42:17.715406 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.715381    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:42:17.814819 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814716    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e311e144-11f5-49b5-b4ff-49b5072889a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.814819 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814756    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.814819 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814775    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e311e144-11f5-49b5-b4ff-49b5072889a6-config-out\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.814819 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814800    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e311e144-11f5-49b5-b4ff-49b5072889a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814833    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e311e144-11f5-49b5-b4ff-49b5072889a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e311e144-11f5-49b5-b4ff-49b5072889a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814898    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814916    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814933    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c98mm\" (UniqueName: \"kubernetes.io/projected/e311e144-11f5-49b5-b4ff-49b5072889a6-kube-api-access-c98mm\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.814984    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-web-config\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.815043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.815079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.815110 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.815105    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.830676 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.830644    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2" path="/var/lib/kubelet/pods/4656a07f-62a5-4600-ba7e-f2e9d1e1d0c2/volumes"
Sep 17 15:42:17.916235 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916196    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916242    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916262    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c98mm\" (UniqueName: \"kubernetes.io/projected/e311e144-11f5-49b5-b4ff-49b5072889a6-kube-api-access-c98mm\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916287    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-web-config\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916311    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916331    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916366 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916644 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916514    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e311e144-11f5-49b5-b4ff-49b5072889a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916644 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916540    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916644 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916570    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e311e144-11f5-49b5-b4ff-49b5072889a6-config-out\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916644 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916594    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e311e144-11f5-49b5-b4ff-49b5072889a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916644 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e311e144-11f5-49b5-b4ff-49b5072889a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.916987 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.916656    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e311e144-11f5-49b5-b4ff-49b5072889a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.917786 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.917755    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e311e144-11f5-49b5-b4ff-49b5072889a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.917786 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.917768    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e311e144-11f5-49b5-b4ff-49b5072889a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.918575 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.918549    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e311e144-11f5-49b5-b4ff-49b5072889a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.919367 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.919345    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.919712 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.919593    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.919712 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.919654    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.919890 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.919857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.919957 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.919925    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.919995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.919961    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.920082 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.920066    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e311e144-11f5-49b5-b4ff-49b5072889a6-web-config\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.920545 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.920521    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e311e144-11f5-49b5-b4ff-49b5072889a6-config-out\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.920800 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.920784    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e311e144-11f5-49b5-b4ff-49b5072889a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:17.926148 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:17.926127    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c98mm\" (UniqueName: \"kubernetes.io/projected/e311e144-11f5-49b5-b4ff-49b5072889a6-kube-api-access-c98mm\") pod \"alertmanager-main-0\" (UID: \"e311e144-11f5-49b5-b4ff-49b5072889a6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:18.005194 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:18.005156    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:42:18.290979 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:18.290949    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:42:18.293287 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:42:18.293258    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode311e144_11f5_49b5_b4ff_49b5072889a6.slice/crio-e75b45b179e6c33e24821365903a94b07c8994b5c83b603568fd78a9a069c1c9 WatchSource:0}: Error finding container e75b45b179e6c33e24821365903a94b07c8994b5c83b603568fd78a9a069c1c9: Status 404 returned error can't find the container with id e75b45b179e6c33e24821365903a94b07c8994b5c83b603568fd78a9a069c1c9
Sep 17 15:42:18.629262 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:18.629177    2568 generic.go:356] "Generic (PLEG): container finished" podID="e311e144-11f5-49b5-b4ff-49b5072889a6" containerID="107c6e6c6cd60c46010ae2004b4b09a0af10b1ae99333ada26273f67d48f8b97" exitCode=0
Sep 17 15:42:18.629708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:18.629257    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerDied","Data":"107c6e6c6cd60c46010ae2004b4b09a0af10b1ae99333ada26273f67d48f8b97"}
Sep 17 15:42:18.629708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:18.629332    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerStarted","Data":"e75b45b179e6c33e24821365903a94b07c8994b5c83b603568fd78a9a069c1c9"}
Sep 17 15:42:19.638229 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:19.638140    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerStarted","Data":"d8c4703d86b0f7408404793806b3314e3ea3dcf840068199f3bd462fe1f51b5c"}
Sep 17 15:42:19.638229 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:19.638205    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerStarted","Data":"2734f7b55f229bbe15790d6be5b8dbfd3df6a66d9d91996980b28d64b3712505"}
Sep 17 15:42:19.638229 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:19.638230    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerStarted","Data":"35261f874f70289e64327072a50796a625b948ea74889680ad2778b00e799635"}
Sep 17 15:42:19.638665 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:19.638258    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerStarted","Data":"dcc4f0842e7194f56155d377717ca422527808069e4992d8b4065f1aefc57e95"}
Sep 17 15:42:19.638665 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:19.638287    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerStarted","Data":"d8c8d07bb3c700247cb13ac5d038119f9b04ac69c6cb91fae2c8f5e5ed55a174"}
Sep 17 15:42:19.638665 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:19.638314    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e311e144-11f5-49b5-b4ff-49b5072889a6","Type":"ContainerStarted","Data":"a5a5db098e2c12e4c84de0dbbb37751f520d7b5632ba6586433a6c28682a08c6"}
Sep 17 15:42:19.677794 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:19.677654    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.657887963 podStartE2EDuration="2.677633152s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.745189ms" isStatefulPod=false podCreationTimestamp="2026-09-17 15:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:42:19.676169181 +0000 UTC m=+184.264213768" watchObservedRunningTime="2026-09-17 15:42:19.677633152 +0000 UTC m=+184.265677739"
Sep 17 15:42:49.990757 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:49.990717    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cc89dfbcc-d4t86"]
Sep 17 15:42:49.994365 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:49.994342    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.007577 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.007552    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc89dfbcc-d4t86"]
Sep 17 15:42:50.091762 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.091731    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-oauth-serving-cert\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.091967 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.091770    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-serving-cert\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.091967 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.091792    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-trusted-ca-bundle\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.091967 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.091857    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkj9f\" (UniqueName: \"kubernetes.io/projected/81aebaa9-555b-40ce-927b-4f9fa6af6668-kube-api-access-nkj9f\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.091967 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.091940    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-service-ca\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.092146 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.091976    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-config\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.092146 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.092004    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-oauth-config\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.192975 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.192938    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-oauth-serving-cert\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.192989    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-serving-cert\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193051    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-trusted-ca-bundle\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkj9f\" (UniqueName: \"kubernetes.io/projected/81aebaa9-555b-40ce-927b-4f9fa6af6668-kube-api-access-nkj9f\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193224 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193156    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-service-ca\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193224 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193194    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-config\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193321 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193224    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-oauth-config\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193742 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193711    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-oauth-serving-cert\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.193983 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193956    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-config\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.194037 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193964    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-trusted-ca-bundle\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.194037 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.193964    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-service-ca\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.195527 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.195508    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-oauth-config\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.195694 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.195675    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-serving-cert\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.203265 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.203246    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkj9f\" (UniqueName: \"kubernetes.io/projected/81aebaa9-555b-40ce-927b-4f9fa6af6668-kube-api-access-nkj9f\") pod \"console-5cc89dfbcc-d4t86\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") " pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.304556 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.304462    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:42:50.579279 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.579254    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc89dfbcc-d4t86"]
Sep 17 15:42:50.581483 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:42:50.581452    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81aebaa9_555b_40ce_927b_4f9fa6af6668.slice/crio-2a07ba3e41c041d711896c1aeed3c2968f25a4de7ae34d4e0146a7ec3c4a7731 WatchSource:0}: Error finding container 2a07ba3e41c041d711896c1aeed3c2968f25a4de7ae34d4e0146a7ec3c4a7731: Status 404 returned error can't find the container with id 2a07ba3e41c041d711896c1aeed3c2968f25a4de7ae34d4e0146a7ec3c4a7731
Sep 17 15:42:50.766927 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.766886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc89dfbcc-d4t86" event={"ID":"81aebaa9-555b-40ce-927b-4f9fa6af6668","Type":"ContainerStarted","Data":"23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7"}
Sep 17 15:42:50.767087 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.766945    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc89dfbcc-d4t86" event={"ID":"81aebaa9-555b-40ce-927b-4f9fa6af6668","Type":"ContainerStarted","Data":"2a07ba3e41c041d711896c1aeed3c2968f25a4de7ae34d4e0146a7ec3c4a7731"}
Sep 17 15:42:50.790287 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:42:50.790233    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cc89dfbcc-d4t86" podStartSLOduration=1.7902167009999999 podStartE2EDuration="1.790216701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:42:50.789249953 +0000 UTC m=+215.377294542" watchObservedRunningTime="2026-09-17 15:42:50.790216701 +0000 UTC m=+215.378261288"
Sep 17 15:43:00.305576 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:00.305545    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:43:00.306117 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:00.305807    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:43:00.310871 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:00.310833    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:43:00.807733 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:00.807706    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:43:00.872935 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:00.872903    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c7d8dc7b7-rqqvs"]
Sep 17 15:43:25.893920 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:25.893856    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c7d8dc7b7-rqqvs" podUID="0e033ad9-34f4-4029-9d01-03252473bdcc" containerName="console" containerID="cri-o://5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34" gracePeriod=15
Sep 17 15:43:26.132119 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.132091    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:43:26.299468 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.299434    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-serving-cert\") pod \"0e033ad9-34f4-4029-9d01-03252473bdcc\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") "
Sep 17 15:43:26.299468 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.299474    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-oauth-config\") pod \"0e033ad9-34f4-4029-9d01-03252473bdcc\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") "
Sep 17 15:43:26.299674 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.299500    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lqxc\" (UniqueName: \"kubernetes.io/projected/0e033ad9-34f4-4029-9d01-03252473bdcc-kube-api-access-4lqxc\") pod \"0e033ad9-34f4-4029-9d01-03252473bdcc\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") "
Sep 17 15:43:26.299674 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.299626    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-trusted-ca-bundle\") pod \"0e033ad9-34f4-4029-9d01-03252473bdcc\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") "
Sep 17 15:43:26.299750 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.299714    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-console-config\") pod \"0e033ad9-34f4-4029-9d01-03252473bdcc\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") "
Sep 17 15:43:26.299820 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.299795    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-oauth-serving-cert\") pod \"0e033ad9-34f4-4029-9d01-03252473bdcc\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") "
Sep 17 15:43:26.299960 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.299879    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-service-ca\") pod \"0e033ad9-34f4-4029-9d01-03252473bdcc\" (UID: \"0e033ad9-34f4-4029-9d01-03252473bdcc\") "
Sep 17 15:43:26.300068 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.300036    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e033ad9-34f4-4029-9d01-03252473bdcc" (UID: "0e033ad9-34f4-4029-9d01-03252473bdcc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:43:26.300277 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.300101    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-console-config" (OuterVolumeSpecName: "console-config") pod "0e033ad9-34f4-4029-9d01-03252473bdcc" (UID: "0e033ad9-34f4-4029-9d01-03252473bdcc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:43:26.300277 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.300217    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-trusted-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:43:26.300376 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.300299    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e033ad9-34f4-4029-9d01-03252473bdcc" (UID: "0e033ad9-34f4-4029-9d01-03252473bdcc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:43:26.300376 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.300335    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e033ad9-34f4-4029-9d01-03252473bdcc" (UID: "0e033ad9-34f4-4029-9d01-03252473bdcc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:43:26.301752 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.301730    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e033ad9-34f4-4029-9d01-03252473bdcc-kube-api-access-4lqxc" (OuterVolumeSpecName: "kube-api-access-4lqxc") pod "0e033ad9-34f4-4029-9d01-03252473bdcc" (UID: "0e033ad9-34f4-4029-9d01-03252473bdcc"). InnerVolumeSpecName "kube-api-access-4lqxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:43:26.302055 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.302032    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e033ad9-34f4-4029-9d01-03252473bdcc" (UID: "0e033ad9-34f4-4029-9d01-03252473bdcc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:43:26.302055 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.302043    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e033ad9-34f4-4029-9d01-03252473bdcc" (UID: "0e033ad9-34f4-4029-9d01-03252473bdcc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:43:26.401445 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.401417    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-console-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:43:26.401445 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.401440    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-oauth-serving-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:43:26.401445 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.401450    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e033ad9-34f4-4029-9d01-03252473bdcc-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:43:26.401672 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.401459    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-serving-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:43:26.401672 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.401467    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e033ad9-34f4-4029-9d01-03252473bdcc-console-oauth-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:43:26.401672 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.401475    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lqxc\" (UniqueName: \"kubernetes.io/projected/0e033ad9-34f4-4029-9d01-03252473bdcc-kube-api-access-4lqxc\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:43:26.910584 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.910546    2568 generic.go:356] "Generic (PLEG): container finished" podID="0e033ad9-34f4-4029-9d01-03252473bdcc" containerID="5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34" exitCode=0
Sep 17 15:43:26.910584 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.910588    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7d8dc7b7-rqqvs"
Sep 17 15:43:26.911130 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.910631    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7d8dc7b7-rqqvs" event={"ID":"0e033ad9-34f4-4029-9d01-03252473bdcc","Type":"ContainerDied","Data":"5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34"}
Sep 17 15:43:26.911130 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.910696    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7d8dc7b7-rqqvs" event={"ID":"0e033ad9-34f4-4029-9d01-03252473bdcc","Type":"ContainerDied","Data":"44054f147ab29ed6a37736c2cdc48c84db4f7607e8ab1c1bb5481f0f63a37566"}
Sep 17 15:43:26.911130 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.910729    2568 scope.go:117] "RemoveContainer" containerID="5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34"
Sep 17 15:43:26.919679 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.919660    2568 scope.go:117] "RemoveContainer" containerID="5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34"
Sep 17 15:43:26.919978 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:43:26.919959    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34\": container with ID starting with 5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34 not found: ID does not exist" containerID="5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34"
Sep 17 15:43:26.920048 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.919985    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34"} err="failed to get container status \"5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34\": rpc error: code = NotFound desc = could not find container \"5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34\": container with ID starting with 5875e027f6064b5878f5d2046e06d8232f16f7b54cbc2ad3f185914f7dfb2e34 not found: ID does not exist"
Sep 17 15:43:26.934283 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.934251    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c7d8dc7b7-rqqvs"]
Sep 17 15:43:26.938104 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:26.938081    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c7d8dc7b7-rqqvs"]
Sep 17 15:43:27.831046 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:43:27.831012    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e033ad9-34f4-4029-9d01-03252473bdcc" path="/var/lib/kubelet/pods/0e033ad9-34f4-4029-9d01-03252473bdcc/volumes"
Sep 17 15:44:15.756157 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:15.756123    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:44:15.759555 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:15.759526    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:44:15.766716 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:15.766693    2568 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 15:44:28.982428 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.982379    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-fkcvj"]
Sep 17 15:44:28.984935 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.982879    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e033ad9-34f4-4029-9d01-03252473bdcc" containerName="console"
Sep 17 15:44:28.984935 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.982899    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e033ad9-34f4-4029-9d01-03252473bdcc" containerName="console"
Sep 17 15:44:28.984935 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.982983    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e033ad9-34f4-4029-9d01-03252473bdcc" containerName="console"
Sep 17 15:44:28.985854 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.985822    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:28.992267 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.991895    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 17 15:44:28.992267 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.991902    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 15:44:28.992557 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.992539    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 17 15:44:28.992621 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.992552    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 15:44:28.992621 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.992546    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-8jmgk\""
Sep 17 15:44:28.992854 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.992823    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 15:44:28.998955 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:28.998933    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-fkcvj"]
Sep 17 15:44:29.108657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.108621    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/41a23326-6f8f-48d3-887f-5fe764d016ba-cabundle0\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.108657 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.108659    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.108881 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.108745    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl4ms\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-kube-api-access-wl4ms\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.209822 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.209789    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/41a23326-6f8f-48d3-887f-5fe764d016ba-cabundle0\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.209955 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.209861    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.209955 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.209913    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl4ms\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-kube-api-access-wl4ms\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.210066 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.210041    2568 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Sep 17 15:44:29.210119 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.210069    2568 secret.go:281] references non-existent secret key: ca.crt
Sep 17 15:44:29.210119 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.210081    2568 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 15:44:29.210119 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.210098    2568 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-fkcvj: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 15:44:29.210274 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.210164    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates podName:41a23326-6f8f-48d3-887f-5fe764d016ba nodeName:}" failed. No retries permitted until 2026-09-17 15:44:29.710144282 +0000 UTC m=+314.298188849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates") pod "keda-operator-579c8958f7-fkcvj" (UID: "41a23326-6f8f-48d3-887f-5fe764d016ba") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 15:44:29.210413 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.210392    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/41a23326-6f8f-48d3-887f-5fe764d016ba-cabundle0\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.222975 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.222956    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl4ms\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-kube-api-access-wl4ms\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.715074 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:29.715028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:29.715276 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.715196    2568 secret.go:281] references non-existent secret key: ca.crt
Sep 17 15:44:29.715276 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.715220    2568 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 15:44:29.715276 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.715230    2568 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-fkcvj: references non-existent secret key: ca.crt
Sep 17 15:44:29.715393 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:44:29.715285    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates podName:41a23326-6f8f-48d3-887f-5fe764d016ba nodeName:}" failed. No retries permitted until 2026-09-17 15:44:30.715269035 +0000 UTC m=+315.303313600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates") pod "keda-operator-579c8958f7-fkcvj" (UID: "41a23326-6f8f-48d3-887f-5fe764d016ba") : references non-existent secret key: ca.crt
Sep 17 15:44:30.726344 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:30.726301    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:30.729018 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:30.728982    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/41a23326-6f8f-48d3-887f-5fe764d016ba-certificates\") pod \"keda-operator-579c8958f7-fkcvj\" (UID: \"41a23326-6f8f-48d3-887f-5fe764d016ba\") " pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:30.795234 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:30.795194    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:31.074853 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:31.074763    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-fkcvj"]
Sep 17 15:44:31.078240 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:44:31.078206    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a23326_6f8f_48d3_887f_5fe764d016ba.slice/crio-16e37ad470186c37057359ba23c67547f1ea3e9594c2d8e2cb0d7cb5e776c484 WatchSource:0}: Error finding container 16e37ad470186c37057359ba23c67547f1ea3e9594c2d8e2cb0d7cb5e776c484: Status 404 returned error can't find the container with id 16e37ad470186c37057359ba23c67547f1ea3e9594c2d8e2cb0d7cb5e776c484
Sep 17 15:44:31.079746 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:31.079725    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 15:44:31.160323 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:31.160282    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-fkcvj" event={"ID":"41a23326-6f8f-48d3-887f-5fe764d016ba","Type":"ContainerStarted","Data":"16e37ad470186c37057359ba23c67547f1ea3e9594c2d8e2cb0d7cb5e776c484"}
Sep 17 15:44:35.182601 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:35.182510    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-fkcvj" event={"ID":"41a23326-6f8f-48d3-887f-5fe764d016ba","Type":"ContainerStarted","Data":"99d93df4ea12dc2fe765bef9cb3e48700680b37a80feb8093dac94370d2d2f79"}
Sep 17 15:44:35.182601 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:35.182592    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:44:35.203470 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:35.203418    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-fkcvj" podStartSLOduration=3.586916213 podStartE2EDuration="7.203402956s" totalImagesPullingTime="3.616486743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:44:35.20268771 +0000 UTC m=+319.790732294" watchObservedRunningTime="2026-09-17 15:44:35.203402956 +0000 UTC m=+319.791447583"
Sep 17 15:44:56.189247 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:44:56.189208    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-fkcvj"
Sep 17 15:45:51.352816 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.352783    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-rq4l7"]
Sep 17 15:45:51.355505 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.355485    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:51.357887 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.357867    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 17 15:45:51.358333 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.358316    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 15:45:51.358753 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.358736    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-6r8dx\""
Sep 17 15:45:51.359116 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.359102    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 15:45:51.374732 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.374709    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-rq4l7"]
Sep 17 15:45:51.416626 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.416597    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-mtqfj"]
Sep 17 15:45:51.418972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.418955    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.421494 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.421467    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 17 15:45:51.421627 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.421608    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-t55m2\""
Sep 17 15:45:51.432567 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.432545    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-mtqfj"]
Sep 17 15:45:51.480653 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.480620    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfzjm\" (UniqueName: \"kubernetes.io/projected/64d39f89-cfa9-4471-a88b-2e45e107a04f-kube-api-access-vfzjm\") pod \"kserve-controller-manager-74b7f9c78b-rq4l7\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") " pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:51.480830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.480668    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert\") pod \"kserve-controller-manager-74b7f9c78b-rq4l7\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") " pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:51.480830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.480687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzrpz\" (UniqueName: \"kubernetes.io/projected/c97d432a-88ef-4a91-a2cf-1c369c5864a4-kube-api-access-lzrpz\") pod \"seaweedfs-64568bcd49-mtqfj\" (UID: \"c97d432a-88ef-4a91-a2cf-1c369c5864a4\") " pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.480830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.480764    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c97d432a-88ef-4a91-a2cf-1c369c5864a4-data\") pod \"seaweedfs-64568bcd49-mtqfj\" (UID: \"c97d432a-88ef-4a91-a2cf-1c369c5864a4\") " pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.582192 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.582139    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfzjm\" (UniqueName: \"kubernetes.io/projected/64d39f89-cfa9-4471-a88b-2e45e107a04f-kube-api-access-vfzjm\") pod \"kserve-controller-manager-74b7f9c78b-rq4l7\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") " pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:51.582192 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.582199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert\") pod \"kserve-controller-manager-74b7f9c78b-rq4l7\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") " pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:51.582377 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.582217    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzrpz\" (UniqueName: \"kubernetes.io/projected/c97d432a-88ef-4a91-a2cf-1c369c5864a4-kube-api-access-lzrpz\") pod \"seaweedfs-64568bcd49-mtqfj\" (UID: \"c97d432a-88ef-4a91-a2cf-1c369c5864a4\") " pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.582377 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.582265    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c97d432a-88ef-4a91-a2cf-1c369c5864a4-data\") pod \"seaweedfs-64568bcd49-mtqfj\" (UID: \"c97d432a-88ef-4a91-a2cf-1c369c5864a4\") " pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.582377 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:45:51.582344    2568 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 17 15:45:51.582491 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:45:51.582421    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert podName:64d39f89-cfa9-4471-a88b-2e45e107a04f nodeName:}" failed. No retries permitted until 2026-09-17 15:45:52.082400819 +0000 UTC m=+396.670445385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert") pod "kserve-controller-manager-74b7f9c78b-rq4l7" (UID: "64d39f89-cfa9-4471-a88b-2e45e107a04f") : secret "kserve-webhook-server-cert" not found
Sep 17 15:45:51.582647 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.582630    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c97d432a-88ef-4a91-a2cf-1c369c5864a4-data\") pod \"seaweedfs-64568bcd49-mtqfj\" (UID: \"c97d432a-88ef-4a91-a2cf-1c369c5864a4\") " pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.596118 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.596088    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfzjm\" (UniqueName: \"kubernetes.io/projected/64d39f89-cfa9-4471-a88b-2e45e107a04f-kube-api-access-vfzjm\") pod \"kserve-controller-manager-74b7f9c78b-rq4l7\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") " pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:51.598410 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.598393    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzrpz\" (UniqueName: \"kubernetes.io/projected/c97d432a-88ef-4a91-a2cf-1c369c5864a4-kube-api-access-lzrpz\") pod \"seaweedfs-64568bcd49-mtqfj\" (UID: \"c97d432a-88ef-4a91-a2cf-1c369c5864a4\") " pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.727673 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.727637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:51.999986 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:51.999960    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-mtqfj"]
Sep 17 15:45:52.002573 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:45:52.002545    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97d432a_88ef_4a91_a2cf_1c369c5864a4.slice/crio-ccdce57a5ebb2fe714a7ad6dbc607c6109da61ba6d57f8a29756686e9d8467f0 WatchSource:0}: Error finding container ccdce57a5ebb2fe714a7ad6dbc607c6109da61ba6d57f8a29756686e9d8467f0: Status 404 returned error can't find the container with id ccdce57a5ebb2fe714a7ad6dbc607c6109da61ba6d57f8a29756686e9d8467f0
Sep 17 15:45:52.087150 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:52.087120    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert\") pod \"kserve-controller-manager-74b7f9c78b-rq4l7\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") " pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:52.089447 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:52.089428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert\") pod \"kserve-controller-manager-74b7f9c78b-rq4l7\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") " pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:52.265934 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:52.265832    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:52.485211 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:52.485178    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-mtqfj" event={"ID":"c97d432a-88ef-4a91-a2cf-1c369c5864a4","Type":"ContainerStarted","Data":"ccdce57a5ebb2fe714a7ad6dbc607c6109da61ba6d57f8a29756686e9d8467f0"}
Sep 17 15:45:52.535884 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:52.535864    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-rq4l7"]
Sep 17 15:45:52.537802 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:45:52.537774    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64d39f89_cfa9_4471_a88b_2e45e107a04f.slice/crio-724e5c17ec38a7731b6297ac8a5c98493fcfd4d3dd03b34d09f3ed913cf3f876 WatchSource:0}: Error finding container 724e5c17ec38a7731b6297ac8a5c98493fcfd4d3dd03b34d09f3ed913cf3f876: Status 404 returned error can't find the container with id 724e5c17ec38a7731b6297ac8a5c98493fcfd4d3dd03b34d09f3ed913cf3f876
Sep 17 15:45:53.491431 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:53.491392    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7" event={"ID":"64d39f89-cfa9-4471-a88b-2e45e107a04f","Type":"ContainerStarted","Data":"724e5c17ec38a7731b6297ac8a5c98493fcfd4d3dd03b34d09f3ed913cf3f876"}
Sep 17 15:45:55.506485 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:55.506389    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-mtqfj" event={"ID":"c97d432a-88ef-4a91-a2cf-1c369c5864a4","Type":"ContainerStarted","Data":"cc97e4f27da70c58d56e45a14a0faa88b8eb61975074ed9bd5255c2edf2b0e16"}
Sep 17 15:45:55.506981 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:55.506562    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:45:55.528430 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:55.528375    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-mtqfj" podStartSLOduration=1.477064487 podStartE2EDuration="4.528360597s" totalImagesPullingTime="3.05129611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:45:55.526923871 +0000 UTC m=+400.114968460" watchObservedRunningTime="2026-09-17 15:45:55.528360597 +0000 UTC m=+400.116405184"
Sep 17 15:45:57.515990 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:57.515959    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7" event={"ID":"64d39f89-cfa9-4471-a88b-2e45e107a04f","Type":"ContainerStarted","Data":"5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149"}
Sep 17 15:45:57.516365 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:57.516075    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:45:57.542042 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:45:57.541997    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7" podStartSLOduration=2.111954199 podStartE2EDuration="6.54198109s" totalImagesPullingTime="4.430026891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:45:57.540405294 +0000 UTC m=+402.128449882" watchObservedRunningTime="2026-09-17 15:45:57.54198109 +0000 UTC m=+402.130025674"
Sep 17 15:46:01.512938 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:01.512900    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-mtqfj"
Sep 17 15:46:24.856452 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.856417    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56c584c7cd-n9gxz"]
Sep 17 15:46:24.862714 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.862690    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.868561 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.868528    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd342800-5257-48b0-85ef-419922ac14be-console-serving-cert\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.868708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.868577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-trusted-ca-bundle\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.868708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.868607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwzkk\" (UniqueName: \"kubernetes.io/projected/fd342800-5257-48b0-85ef-419922ac14be-kube-api-access-zwzkk\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.868708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.868680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-console-config\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.868824 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.868709    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-service-ca\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.868824 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.868749    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd342800-5257-48b0-85ef-419922ac14be-console-oauth-config\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.868824 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.868771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-oauth-serving-cert\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.875477 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.875453    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c584c7cd-n9gxz"]
Sep 17 15:46:24.969653 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.969607    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd342800-5257-48b0-85ef-419922ac14be-console-oauth-config\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.969653 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.969656    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-oauth-serving-cert\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.969972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.969688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd342800-5257-48b0-85ef-419922ac14be-console-serving-cert\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.969972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.969703    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-trusted-ca-bundle\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.969972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.969720    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwzkk\" (UniqueName: \"kubernetes.io/projected/fd342800-5257-48b0-85ef-419922ac14be-kube-api-access-zwzkk\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.969972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.969768    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-console-config\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.969972 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.969804    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-service-ca\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.970622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.970594    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-service-ca\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.970711 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.970593    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-oauth-serving-cert\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.970711 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.970603    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-trusted-ca-bundle\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.970711 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.970677    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd342800-5257-48b0-85ef-419922ac14be-console-config\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.972102 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.972084    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd342800-5257-48b0-85ef-419922ac14be-console-serving-cert\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.972188 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.972158    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd342800-5257-48b0-85ef-419922ac14be-console-oauth-config\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:24.979278 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:24.979248    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwzkk\" (UniqueName: \"kubernetes.io/projected/fd342800-5257-48b0-85ef-419922ac14be-kube-api-access-zwzkk\") pod \"console-56c584c7cd-n9gxz\" (UID: \"fd342800-5257-48b0-85ef-419922ac14be\") " pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:25.172229 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:25.172142    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:25.453995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:25.453963    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c584c7cd-n9gxz"]
Sep 17 15:46:25.457072 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:46:25.457038    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd342800_5257_48b0_85ef_419922ac14be.slice/crio-8bfd080fd6129187885ae63a0fe4d4111ceaa2968753a1b1df19381cbe0065ce WatchSource:0}: Error finding container 8bfd080fd6129187885ae63a0fe4d4111ceaa2968753a1b1df19381cbe0065ce: Status 404 returned error can't find the container with id 8bfd080fd6129187885ae63a0fe4d4111ceaa2968753a1b1df19381cbe0065ce
Sep 17 15:46:25.629147 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:25.629111    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c584c7cd-n9gxz" event={"ID":"fd342800-5257-48b0-85ef-419922ac14be","Type":"ContainerStarted","Data":"2da25207d98bbe30ccaecd13eed813247cba1b3ba40eaaef28a7c5205e42b5de"}
Sep 17 15:46:25.629307 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:25.629164    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c584c7cd-n9gxz" event={"ID":"fd342800-5257-48b0-85ef-419922ac14be","Type":"ContainerStarted","Data":"8bfd080fd6129187885ae63a0fe4d4111ceaa2968753a1b1df19381cbe0065ce"}
Sep 17 15:46:25.655990 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:25.655934    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56c584c7cd-n9gxz" podStartSLOduration=1.6559143189999999 podStartE2EDuration="1.655914319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:46:25.654907727 +0000 UTC m=+430.242952308" watchObservedRunningTime="2026-09-17 15:46:25.655914319 +0000 UTC m=+430.243958897"
Sep 17 15:46:28.525908 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:28.525876    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:46:35.073327 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.073291    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-rq4l7"]
Sep 17 15:46:35.073782 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.073483    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7" podUID="64d39f89-cfa9-4471-a88b-2e45e107a04f" containerName="manager" containerID="cri-o://5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149" gracePeriod=10
Sep 17 15:46:35.111016 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.110980    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-vpmlb"]
Sep 17 15:46:35.113441 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.113425    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.126330 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.126307    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-vpmlb"]
Sep 17 15:46:35.141095 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.141068    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda-cert\") pod \"kserve-controller-manager-74b7f9c78b-vpmlb\" (UID: \"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda\") " pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.141228 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.141108    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm9vt\" (UniqueName: \"kubernetes.io/projected/473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda-kube-api-access-vm9vt\") pod \"kserve-controller-manager-74b7f9c78b-vpmlb\" (UID: \"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda\") " pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.172818 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.172789    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:35.172944 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.172894    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:35.177729 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.177709    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:35.241617 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.241588    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda-cert\") pod \"kserve-controller-manager-74b7f9c78b-vpmlb\" (UID: \"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda\") " pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.241775 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.241730    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm9vt\" (UniqueName: \"kubernetes.io/projected/473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda-kube-api-access-vm9vt\") pod \"kserve-controller-manager-74b7f9c78b-vpmlb\" (UID: \"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda\") " pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.244074 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.244052    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda-cert\") pod \"kserve-controller-manager-74b7f9c78b-vpmlb\" (UID: \"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda\") " pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.254454 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.254432    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm9vt\" (UniqueName: \"kubernetes.io/projected/473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda-kube-api-access-vm9vt\") pod \"kserve-controller-manager-74b7f9c78b-vpmlb\" (UID: \"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda\") " pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.309370 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.309346    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:46:35.342970 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.342891    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfzjm\" (UniqueName: \"kubernetes.io/projected/64d39f89-cfa9-4471-a88b-2e45e107a04f-kube-api-access-vfzjm\") pod \"64d39f89-cfa9-4471-a88b-2e45e107a04f\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") "
Sep 17 15:46:35.343114 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.342984    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert\") pod \"64d39f89-cfa9-4471-a88b-2e45e107a04f\" (UID: \"64d39f89-cfa9-4471-a88b-2e45e107a04f\") "
Sep 17 15:46:35.345213 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.345178    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert" (OuterVolumeSpecName: "cert") pod "64d39f89-cfa9-4471-a88b-2e45e107a04f" (UID: "64d39f89-cfa9-4471-a88b-2e45e107a04f"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:46:35.345371 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.345348    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64d39f89-cfa9-4471-a88b-2e45e107a04f-kube-api-access-vfzjm" (OuterVolumeSpecName: "kube-api-access-vfzjm") pod "64d39f89-cfa9-4471-a88b-2e45e107a04f" (UID: "64d39f89-cfa9-4471-a88b-2e45e107a04f"). InnerVolumeSpecName "kube-api-access-vfzjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:46:35.443647 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.443606    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfzjm\" (UniqueName: \"kubernetes.io/projected/64d39f89-cfa9-4471-a88b-2e45e107a04f-kube-api-access-vfzjm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:46:35.443647 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.443640    2568 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64d39f89-cfa9-4471-a88b-2e45e107a04f-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:46:35.456539 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.456516    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:35.673359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.673270    2568 generic.go:356] "Generic (PLEG): container finished" podID="64d39f89-cfa9-4471-a88b-2e45e107a04f" containerID="5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149" exitCode=0
Sep 17 15:46:35.673359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.673321    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7" event={"ID":"64d39f89-cfa9-4471-a88b-2e45e107a04f","Type":"ContainerDied","Data":"5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149"}
Sep 17 15:46:35.673359 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.673335    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7"
Sep 17 15:46:35.673611 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.673376    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74b7f9c78b-rq4l7" event={"ID":"64d39f89-cfa9-4471-a88b-2e45e107a04f","Type":"ContainerDied","Data":"724e5c17ec38a7731b6297ac8a5c98493fcfd4d3dd03b34d09f3ed913cf3f876"}
Sep 17 15:46:35.673611 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.673404    2568 scope.go:117] "RemoveContainer" containerID="5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149"
Sep 17 15:46:35.677983 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.677958    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56c584c7cd-n9gxz"
Sep 17 15:46:35.682346 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.682330    2568 scope.go:117] "RemoveContainer" containerID="5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149"
Sep 17 15:46:35.682621 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:46:35.682603    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149\": container with ID starting with 5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149 not found: ID does not exist" containerID="5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149"
Sep 17 15:46:35.682671 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.682632    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149"} err="failed to get container status \"5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149\": rpc error: code = NotFound desc = could not find container \"5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149\": container with ID starting with 5f3cc308f5dd6f309cdde8589ccec2166ccc185eae8cc7dad944676cd8144149 not found: ID does not exist"
Sep 17 15:46:35.720302 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.720269    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-rq4l7"]
Sep 17 15:46:35.730970 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.729558    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-rq4l7"]
Sep 17 15:46:35.737064 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.737034    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74b7f9c78b-vpmlb"]
Sep 17 15:46:35.739794 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.739769    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cc89dfbcc-d4t86"]
Sep 17 15:46:35.741288 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:46:35.741258    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473bd3f5_cf1c_4ae1_8ea2_7f67c7640dda.slice/crio-cfe99e7884992527474831ca8c2c42bdff1f188421e3ee10419ffa6828e81c25 WatchSource:0}: Error finding container cfe99e7884992527474831ca8c2c42bdff1f188421e3ee10419ffa6828e81c25: Status 404 returned error can't find the container with id cfe99e7884992527474831ca8c2c42bdff1f188421e3ee10419ffa6828e81c25
Sep 17 15:46:35.832231 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:35.832194    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64d39f89-cfa9-4471-a88b-2e45e107a04f" path="/var/lib/kubelet/pods/64d39f89-cfa9-4471-a88b-2e45e107a04f/volumes"
Sep 17 15:46:36.679765 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:36.679723    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb" event={"ID":"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda","Type":"ContainerStarted","Data":"6d0df3bce73228b6945dc644c964d4e62dabb93a3b2ea7170629558f7ca5468e"}
Sep 17 15:46:36.680263 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:36.679785    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb" event={"ID":"473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda","Type":"ContainerStarted","Data":"cfe99e7884992527474831ca8c2c42bdff1f188421e3ee10419ffa6828e81c25"}
Sep 17 15:46:36.680263 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:36.679945    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:46:36.703086 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:46:36.703034    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb" podStartSLOduration=1.274644186 podStartE2EDuration="1.703019225s" totalImagesPullingTime="428.375039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:46:36.700318014 +0000 UTC m=+441.288362602" watchObservedRunningTime="2026-09-17 15:46:36.703019225 +0000 UTC m=+441.291063863"
Sep 17 15:47:00.765622 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:00.765556    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cc89dfbcc-d4t86" podUID="81aebaa9-555b-40ce-927b-4f9fa6af6668" containerName="console" containerID="cri-o://23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7" gracePeriod=15
Sep 17 15:47:00.804649 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:00.804615    2568 patch_prober.go:28] interesting pod/console-5cc89dfbcc-d4t86 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" start-of-body=
Sep 17 15:47:00.804819 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:00.804677    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-5cc89dfbcc-d4t86" podUID="81aebaa9-555b-40ce-927b-4f9fa6af6668" containerName="console" probeResult="failure" output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused"
Sep 17 15:47:01.006596 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.006570    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:47:01.067061 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.066976    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkj9f\" (UniqueName: \"kubernetes.io/projected/81aebaa9-555b-40ce-927b-4f9fa6af6668-kube-api-access-nkj9f\") pod \"81aebaa9-555b-40ce-927b-4f9fa6af6668\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") "
Sep 17 15:47:01.067061 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067032    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-oauth-serving-cert\") pod \"81aebaa9-555b-40ce-927b-4f9fa6af6668\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") "
Sep 17 15:47:01.067261 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067065    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-service-ca\") pod \"81aebaa9-555b-40ce-927b-4f9fa6af6668\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") "
Sep 17 15:47:01.067261 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067159    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-oauth-config\") pod \"81aebaa9-555b-40ce-927b-4f9fa6af6668\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") "
Sep 17 15:47:01.067261 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067194    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-serving-cert\") pod \"81aebaa9-555b-40ce-927b-4f9fa6af6668\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") "
Sep 17 15:47:01.067261 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067227    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-trusted-ca-bundle\") pod \"81aebaa9-555b-40ce-927b-4f9fa6af6668\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") "
Sep 17 15:47:01.067480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067275    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-config\") pod \"81aebaa9-555b-40ce-927b-4f9fa6af6668\" (UID: \"81aebaa9-555b-40ce-927b-4f9fa6af6668\") "
Sep 17 15:47:01.067480 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067452    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-service-ca" (OuterVolumeSpecName: "service-ca") pod "81aebaa9-555b-40ce-927b-4f9fa6af6668" (UID: "81aebaa9-555b-40ce-927b-4f9fa6af6668"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:47:01.067595 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067472    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "81aebaa9-555b-40ce-927b-4f9fa6af6668" (UID: "81aebaa9-555b-40ce-927b-4f9fa6af6668"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:47:01.067722 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067697    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-oauth-serving-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:47:01.067864 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067725    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:47:01.067864 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067713    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "81aebaa9-555b-40ce-927b-4f9fa6af6668" (UID: "81aebaa9-555b-40ce-927b-4f9fa6af6668"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:47:01.067864 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.067728    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-config" (OuterVolumeSpecName: "console-config") pod "81aebaa9-555b-40ce-927b-4f9fa6af6668" (UID: "81aebaa9-555b-40ce-927b-4f9fa6af6668"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:47:01.069283 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.069253    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "81aebaa9-555b-40ce-927b-4f9fa6af6668" (UID: "81aebaa9-555b-40ce-927b-4f9fa6af6668"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:47:01.069367 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.069315    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81aebaa9-555b-40ce-927b-4f9fa6af6668-kube-api-access-nkj9f" (OuterVolumeSpecName: "kube-api-access-nkj9f") pod "81aebaa9-555b-40ce-927b-4f9fa6af6668" (UID: "81aebaa9-555b-40ce-927b-4f9fa6af6668"). InnerVolumeSpecName "kube-api-access-nkj9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:47:01.069367 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.069356    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "81aebaa9-555b-40ce-927b-4f9fa6af6668" (UID: "81aebaa9-555b-40ce-927b-4f9fa6af6668"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:47:01.168475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.168435    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:47:01.168475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.168466    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkj9f\" (UniqueName: \"kubernetes.io/projected/81aebaa9-555b-40ce-927b-4f9fa6af6668-kube-api-access-nkj9f\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:47:01.168475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.168478    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-oauth-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:47:01.168475 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.168487    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81aebaa9-555b-40ce-927b-4f9fa6af6668-console-serving-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:47:01.168739 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.168495    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81aebaa9-555b-40ce-927b-4f9fa6af6668-trusted-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:47:01.795275 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.795241    2568 generic.go:356] "Generic (PLEG): container finished" podID="81aebaa9-555b-40ce-927b-4f9fa6af6668" containerID="23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7" exitCode=0
Sep 17 15:47:01.795747 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.795287    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc89dfbcc-d4t86"
Sep 17 15:47:01.795747 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.795291    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc89dfbcc-d4t86" event={"ID":"81aebaa9-555b-40ce-927b-4f9fa6af6668","Type":"ContainerDied","Data":"23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7"}
Sep 17 15:47:01.795747 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.795423    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc89dfbcc-d4t86" event={"ID":"81aebaa9-555b-40ce-927b-4f9fa6af6668","Type":"ContainerDied","Data":"2a07ba3e41c041d711896c1aeed3c2968f25a4de7ae34d4e0146a7ec3c4a7731"}
Sep 17 15:47:01.795747 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.795460    2568 scope.go:117] "RemoveContainer" containerID="23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7"
Sep 17 15:47:01.804025 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.804008    2568 scope.go:117] "RemoveContainer" containerID="23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7"
Sep 17 15:47:01.804313 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:47:01.804291    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7\": container with ID starting with 23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7 not found: ID does not exist" containerID="23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7"
Sep 17 15:47:01.804377 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.804325    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7"} err="failed to get container status \"23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7\": rpc error: code = NotFound desc = could not find container \"23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7\": container with ID starting with 23750b2f33f712026582d2a06e1f265b2f6071120a9e7079da7ca3edd373ebb7 not found: ID does not exist"
Sep 17 15:47:01.820306 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.820275    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cc89dfbcc-d4t86"]
Sep 17 15:47:01.831502 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:01.831475    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cc89dfbcc-d4t86"]
Sep 17 15:47:03.830192 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:03.830153    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81aebaa9-555b-40ce-927b-4f9fa6af6668" path="/var/lib/kubelet/pods/81aebaa9-555b-40ce-927b-4f9fa6af6668/volumes"
Sep 17 15:47:07.690256 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:07.690224    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-74b7f9c78b-vpmlb"
Sep 17 15:47:09.335472 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.335437    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-bnthk"]
Sep 17 15:47:09.335901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.335810    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64d39f89-cfa9-4471-a88b-2e45e107a04f" containerName="manager"
Sep 17 15:47:09.335901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.335824    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="64d39f89-cfa9-4471-a88b-2e45e107a04f" containerName="manager"
Sep 17 15:47:09.335901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.335848    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81aebaa9-555b-40ce-927b-4f9fa6af6668" containerName="console"
Sep 17 15:47:09.335901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.335854    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="81aebaa9-555b-40ce-927b-4f9fa6af6668" containerName="console"
Sep 17 15:47:09.336040 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.335912    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="64d39f89-cfa9-4471-a88b-2e45e107a04f" containerName="manager"
Sep 17 15:47:09.336040 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.335923    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="81aebaa9-555b-40ce-927b-4f9fa6af6668" containerName="console"
Sep 17 15:47:09.338803 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.338786    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.343003 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.342982    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-n569f\""
Sep 17 15:47:09.343145 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.343129    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 17 15:47:09.344275 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.344256    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 17 15:47:09.356720 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.356697    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-bnthk"]
Sep 17 15:47:09.434649 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.434611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96d0667f-ce3b-4705-a7c4-5b5c763efc24-metrics-certs\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.434649 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.434660    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jlj8\" (UniqueName: \"kubernetes.io/projected/96d0667f-ce3b-4705-a7c4-5b5c763efc24-kube-api-access-2jlj8\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.434872 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.434751    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96d0667f-ce3b-4705-a7c4-5b5c763efc24-cert\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.535395 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.535358    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96d0667f-ce3b-4705-a7c4-5b5c763efc24-metrics-certs\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.535395 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.535396    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jlj8\" (UniqueName: \"kubernetes.io/projected/96d0667f-ce3b-4705-a7c4-5b5c763efc24-kube-api-access-2jlj8\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.535605 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.535428    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96d0667f-ce3b-4705-a7c4-5b5c763efc24-cert\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.538048 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.538015    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96d0667f-ce3b-4705-a7c4-5b5c763efc24-metrics-certs\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.538172 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.538058    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96d0667f-ce3b-4705-a7c4-5b5c763efc24-cert\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.553212 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.553178    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jlj8\" (UniqueName: \"kubernetes.io/projected/96d0667f-ce3b-4705-a7c4-5b5c763efc24-kube-api-access-2jlj8\") pod \"odh-model-controller-7b966d7984-bnthk\" (UID: \"96d0667f-ce3b-4705-a7c4-5b5c763efc24\") " pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.649443 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.649355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:09.925758 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:09.925702    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-bnthk"]
Sep 17 15:47:09.929994 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:47:09.929963    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d0667f_ce3b_4705_a7c4_5b5c763efc24.slice/crio-36ebc972a6233eeb10a969fc09441560a8dd3b6109308ded58de0cf8b888a033 WatchSource:0}: Error finding container 36ebc972a6233eeb10a969fc09441560a8dd3b6109308ded58de0cf8b888a033: Status 404 returned error can't find the container with id 36ebc972a6233eeb10a969fc09441560a8dd3b6109308ded58de0cf8b888a033
Sep 17 15:47:10.836470 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:10.836429    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-bnthk" event={"ID":"96d0667f-ce3b-4705-a7c4-5b5c763efc24","Type":"ContainerStarted","Data":"36ebc972a6233eeb10a969fc09441560a8dd3b6109308ded58de0cf8b888a033"}
Sep 17 15:47:13.851064 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:13.851029    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-bnthk" event={"ID":"96d0667f-ce3b-4705-a7c4-5b5c763efc24","Type":"ContainerStarted","Data":"19e7e9fe36698f45fd464977619a1a72d7bce8f93c91ac7ff962cf3646d84c02"}
Sep 17 15:47:13.851476 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:13.851222    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:47:13.889272 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:13.889226    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-bnthk" podStartSLOduration=1.9983281210000001 podStartE2EDuration="4.889211206s" totalImagesPullingTime="2.890883085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:47:13.888971605 +0000 UTC m=+478.477016191" watchObservedRunningTime="2026-09-17 15:47:13.889211206 +0000 UTC m=+478.477255808"
Sep 17 15:47:24.857902 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:47:24.857862    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-bnthk"
Sep 17 15:48:57.172409 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.172370    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"]
Sep 17 15:48:57.176105 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.176087    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.181412 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.181388    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 15:48:57.182499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.182311    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 15:48:57.182499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.182322    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-13ed5-predictor-serving-cert\""
Sep 17 15:48:57.182499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.182272    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-13ed5-kube-rbac-proxy-sar-config\""
Sep 17 15:48:57.182499 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.182352    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9nh5v\""
Sep 17 15:48:57.197425 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.197397    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"]
Sep 17 15:48:57.279920 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.279888    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99qgx\" (UniqueName: \"kubernetes.io/projected/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-api-access-99qgx\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.280087 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.279949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-proxy-tls\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.280087 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.279978    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.380520 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.380484    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99qgx\" (UniqueName: \"kubernetes.io/projected/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-api-access-99qgx\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.380700 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.380565    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-proxy-tls\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.380700 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.380603    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.381329 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.381301    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.383223 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.383202    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-proxy-tls\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.392357 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.392322    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99qgx\" (UniqueName: \"kubernetes.io/projected/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-api-access-99qgx\") pod \"success-200-isvc-13ed5-predictor-78986954d8-cxzxk\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") " pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.486658 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.486615    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:48:57.763462 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:57.763440    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"]
Sep 17 15:48:57.766295 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:48:57.766230    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42548f06_0c80_4f40_bafa_0e3ecd7a0b83.slice/crio-4d9d75af35964c6248c7a3851a9c40935e731bb36ee562b86d4a86c04f2983c8 WatchSource:0}: Error finding container 4d9d75af35964c6248c7a3851a9c40935e731bb36ee562b86d4a86c04f2983c8: Status 404 returned error can't find the container with id 4d9d75af35964c6248c7a3851a9c40935e731bb36ee562b86d4a86c04f2983c8
Sep 17 15:48:58.296789 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:48:58.296746    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" event={"ID":"42548f06-0c80-4f40-bafa-0e3ecd7a0b83","Type":"ContainerStarted","Data":"4d9d75af35964c6248c7a3851a9c40935e731bb36ee562b86d4a86c04f2983c8"}
Sep 17 15:49:10.361392 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:10.361362    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" event={"ID":"42548f06-0c80-4f40-bafa-0e3ecd7a0b83","Type":"ContainerStarted","Data":"5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a"}
Sep 17 15:49:13.380993 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:13.380951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" event={"ID":"42548f06-0c80-4f40-bafa-0e3ecd7a0b83","Type":"ContainerStarted","Data":"95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8"}
Sep 17 15:49:13.381513 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:13.381121    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:49:13.413708 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:13.413656    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podStartSLOduration=1.615164907 podStartE2EDuration="16.413631007s" totalImagesPullingTime="14.7984661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:48:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:49:13.411556429 +0000 UTC m=+597.999601048" watchObservedRunningTime="2026-09-17 15:49:13.413631007 +0000 UTC m=+598.001675593"
Sep 17 15:49:14.385921 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:14.385886    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:49:14.387382 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:14.387350    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 17 15:49:15.390331 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:15.390292    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 17 15:49:15.789709 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:15.789679    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:49:15.790485 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:15.790465    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:49:20.394791 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:20.394759    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:49:20.395438 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:20.395403    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 17 15:49:30.396343 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:30.396303    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 17 15:49:40.396387 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:40.396305    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 17 15:49:50.396312 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:49:50.396271    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 17 15:50:00.396342 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:00.396312    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:50:17.235459 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.235425    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"]
Sep 17 15:50:17.238553 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.238534    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.242337 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.242313    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-13ed5-serving-cert\""
Sep 17 15:50:17.242456 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.242323    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-13ed5-kube-rbac-proxy-sar-config\""
Sep 17 15:50:17.280265 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.280236    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"]
Sep 17 15:50:17.385024 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.384986    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad82845c-be52-4b9a-b04c-84f7ae6127f1-openshift-service-ca-bundle\") pod \"switch-graph-13ed5-64c7cc7cf4-hkn7d\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") " pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.385024 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.385026    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad82845c-be52-4b9a-b04c-84f7ae6127f1-proxy-tls\") pod \"switch-graph-13ed5-64c7cc7cf4-hkn7d\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") " pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.486051 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.485979    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad82845c-be52-4b9a-b04c-84f7ae6127f1-openshift-service-ca-bundle\") pod \"switch-graph-13ed5-64c7cc7cf4-hkn7d\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") " pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.486051 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.486011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad82845c-be52-4b9a-b04c-84f7ae6127f1-proxy-tls\") pod \"switch-graph-13ed5-64c7cc7cf4-hkn7d\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") " pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.486643 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.486621    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad82845c-be52-4b9a-b04c-84f7ae6127f1-openshift-service-ca-bundle\") pod \"switch-graph-13ed5-64c7cc7cf4-hkn7d\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") " pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.488486 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.488462    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad82845c-be52-4b9a-b04c-84f7ae6127f1-proxy-tls\") pod \"switch-graph-13ed5-64c7cc7cf4-hkn7d\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") " pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.548643 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.548613    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:17.823379 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.823350    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"]
Sep 17 15:50:17.825866 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:50:17.825824    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad82845c_be52_4b9a_b04c_84f7ae6127f1.slice/crio-4eddbb3987f1aa0a764516f69dbacf2fc60dc74c737f40e79370c282770ec7d0 WatchSource:0}: Error finding container 4eddbb3987f1aa0a764516f69dbacf2fc60dc74c737f40e79370c282770ec7d0: Status 404 returned error can't find the container with id 4eddbb3987f1aa0a764516f69dbacf2fc60dc74c737f40e79370c282770ec7d0
Sep 17 15:50:17.828194 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:17.828168    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 15:50:18.670126 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:18.670092    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" event={"ID":"ad82845c-be52-4b9a-b04c-84f7ae6127f1","Type":"ContainerStarted","Data":"4eddbb3987f1aa0a764516f69dbacf2fc60dc74c737f40e79370c282770ec7d0"}
Sep 17 15:50:21.683061 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:21.683020    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" event={"ID":"ad82845c-be52-4b9a-b04c-84f7ae6127f1","Type":"ContainerStarted","Data":"2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4"}
Sep 17 15:50:21.683554 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:21.683117    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:21.723348 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:21.723292    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podStartSLOduration=1.877126917 podStartE2EDuration="4.723275506s" totalImagesPullingTime="2.846148589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:50:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:50:21.721872296 +0000 UTC m=+666.309916879" watchObservedRunningTime="2026-09-17 15:50:21.723275506 +0000 UTC m=+666.311320093"
Sep 17 15:50:27.691814 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:27.691786    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:31.329300 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.329256    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"]
Sep 17 15:50:31.329805 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.329501    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5" containerID="cri-o://2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4" gracePeriod=30
Sep 17 15:50:31.540251 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.540217    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"]
Sep 17 15:50:31.540454 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.540429    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container" containerID="cri-o://5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a" gracePeriod=30
Sep 17 15:50:31.540525 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.540486    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kube-rbac-proxy" containerID="cri-o://95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8" gracePeriod=30
Sep 17 15:50:31.728352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.728320    2568 generic.go:356] "Generic (PLEG): container finished" podID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerID="95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8" exitCode=2
Sep 17 15:50:31.728515 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.728386    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" event={"ID":"42548f06-0c80-4f40-bafa-0e3ecd7a0b83","Type":"ContainerDied","Data":"95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8"}
Sep 17 15:50:31.752940 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.752900    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"]
Sep 17 15:50:31.756468 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.756448    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:31.759400 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.759381    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-795b5-kube-rbac-proxy-sar-config\""
Sep 17 15:50:31.759400 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.759399    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-795b5-predictor-serving-cert\""
Sep 17 15:50:31.766216 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.766193    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"]
Sep 17 15:50:31.914485 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.914406    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8039d28-570c-40c4-bb53-0d479af97b96-proxy-tls\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:31.914485 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.914449    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8039d28-570c-40c4-bb53-0d479af97b96-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:31.914485 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:31.914478    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z88x5\" (UniqueName: \"kubernetes.io/projected/d8039d28-570c-40c4-bb53-0d479af97b96-kube-api-access-z88x5\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.015952 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.015916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8039d28-570c-40c4-bb53-0d479af97b96-proxy-tls\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.016131 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.015956    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8039d28-570c-40c4-bb53-0d479af97b96-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.016131 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.015987    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z88x5\" (UniqueName: \"kubernetes.io/projected/d8039d28-570c-40c4-bb53-0d479af97b96-kube-api-access-z88x5\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.016596 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.016571    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8039d28-570c-40c4-bb53-0d479af97b96-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.018352 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.018328    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8039d28-570c-40c4-bb53-0d479af97b96-proxy-tls\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.029830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.029807    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z88x5\" (UniqueName: \"kubernetes.io/projected/d8039d28-570c-40c4-bb53-0d479af97b96-kube-api-access-z88x5\") pod \"success-200-isvc-795b5-predictor-76d9994664-t6lns\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") " pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.068260 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.068236    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.352408 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.352301    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"]
Sep 17 15:50:32.356517 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:50:32.356491    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8039d28_570c_40c4_bb53_0d479af97b96.slice/crio-4575c1a8175f4161633da86d403deaa9413c563d036e8d8f4992751290d56c32 WatchSource:0}: Error finding container 4575c1a8175f4161633da86d403deaa9413c563d036e8d8f4992751290d56c32: Status 404 returned error can't find the container with id 4575c1a8175f4161633da86d403deaa9413c563d036e8d8f4992751290d56c32
Sep 17 15:50:32.690996 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.690960    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:50:32.736122 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.736085    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" event={"ID":"d8039d28-570c-40c4-bb53-0d479af97b96","Type":"ContainerStarted","Data":"470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610"}
Sep 17 15:50:32.736306 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.736152    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" event={"ID":"d8039d28-570c-40c4-bb53-0d479af97b96","Type":"ContainerStarted","Data":"31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb"}
Sep 17 15:50:32.736306 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.736183    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" event={"ID":"d8039d28-570c-40c4-bb53-0d479af97b96","Type":"ContainerStarted","Data":"4575c1a8175f4161633da86d403deaa9413c563d036e8d8f4992751290d56c32"}
Sep 17 15:50:32.736306 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.736246    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:32.767118 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:32.767066    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podStartSLOduration=1.767047715 podStartE2EDuration="1.767047715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:50:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:50:32.76643082 +0000 UTC m=+677.354475408" watchObservedRunningTime="2026-09-17 15:50:32.767047715 +0000 UTC m=+677.355092302"
Sep 17 15:50:33.740905 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:33.740872    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:33.742168 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:33.742139    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 17 15:50:34.589693 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.589671    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:50:34.636760 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.636729    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-rbac-proxy-sar-config\") pod \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") "
Sep 17 15:50:34.636932 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.636779    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-proxy-tls\") pod \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") "
Sep 17 15:50:34.636932 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.636829    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99qgx\" (UniqueName: \"kubernetes.io/projected/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-api-access-99qgx\") pod \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\" (UID: \"42548f06-0c80-4f40-bafa-0e3ecd7a0b83\") "
Sep 17 15:50:34.637166 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.637140    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "42548f06-0c80-4f40-bafa-0e3ecd7a0b83" (UID: "42548f06-0c80-4f40-bafa-0e3ecd7a0b83"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:50:34.639056 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.639025    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "42548f06-0c80-4f40-bafa-0e3ecd7a0b83" (UID: "42548f06-0c80-4f40-bafa-0e3ecd7a0b83"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:50:34.639156 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.639080    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-api-access-99qgx" (OuterVolumeSpecName: "kube-api-access-99qgx") pod "42548f06-0c80-4f40-bafa-0e3ecd7a0b83" (UID: "42548f06-0c80-4f40-bafa-0e3ecd7a0b83"). InnerVolumeSpecName "kube-api-access-99qgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:50:34.738270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.738238    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:50:34.738270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.738267    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99qgx\" (UniqueName: \"kubernetes.io/projected/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-api-access-99qgx\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:50:34.738270 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.738278    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42548f06-0c80-4f40-bafa-0e3ecd7a0b83-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:50:34.747387 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.747355    2568 generic.go:356] "Generic (PLEG): container finished" podID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerID="5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a" exitCode=0
Sep 17 15:50:34.747783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.747390    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" event={"ID":"42548f06-0c80-4f40-bafa-0e3ecd7a0b83","Type":"ContainerDied","Data":"5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a"}
Sep 17 15:50:34.747783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.747427    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"
Sep 17 15:50:34.747783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.747454    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk" event={"ID":"42548f06-0c80-4f40-bafa-0e3ecd7a0b83","Type":"ContainerDied","Data":"4d9d75af35964c6248c7a3851a9c40935e731bb36ee562b86d4a86c04f2983c8"}
Sep 17 15:50:34.747783 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.747486    2568 scope.go:117] "RemoveContainer" containerID="95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8"
Sep 17 15:50:34.748016 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.747993    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 17 15:50:34.755827 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.755810    2568 scope.go:117] "RemoveContainer" containerID="5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a"
Sep 17 15:50:34.765305 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.765283    2568 scope.go:117] "RemoveContainer" containerID="95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8"
Sep 17 15:50:34.765603 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:50:34.765582    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8\": container with ID starting with 95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8 not found: ID does not exist" containerID="95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8"
Sep 17 15:50:34.765670 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.765619    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8"} err="failed to get container status \"95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8\": rpc error: code = NotFound desc = could not find container \"95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8\": container with ID starting with 95b1803b23c2ea0b974ac7ab7582d2ce2f11dbcc952a69ba52c41c08dd72a7c8 not found: ID does not exist"
Sep 17 15:50:34.765670 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.765637    2568 scope.go:117] "RemoveContainer" containerID="5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a"
Sep 17 15:50:34.765929 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:50:34.765907    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a\": container with ID starting with 5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a not found: ID does not exist" containerID="5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a"
Sep 17 15:50:34.765979 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.765936    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a"} err="failed to get container status \"5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a\": rpc error: code = NotFound desc = could not find container \"5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a\": container with ID starting with 5e090953fa24cc3a349056bfe19b9a917b4102826aed63e5143127f95e90cc1a not found: ID does not exist"
Sep 17 15:50:34.776578 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.776553    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"]
Sep 17 15:50:34.785730 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:34.785708    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-13ed5-predictor-78986954d8-cxzxk"]
Sep 17 15:50:35.832086 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:35.832057    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" path="/var/lib/kubelet/pods/42548f06-0c80-4f40-bafa-0e3ecd7a0b83/volumes"
Sep 17 15:50:37.690470 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:37.690433    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:50:39.752992 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:39.752965    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:50:39.753551 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:39.753526    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 17 15:50:42.691452 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:42.691411    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:50:42.691830 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:42.691510    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:50:47.691388 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:47.691351    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:50:49.754245 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:49.754204    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 17 15:50:52.690511 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:52.690474    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:50:57.691374 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:57.691329    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:50:59.754306 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:50:59.754266    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 17 15:51:01.478272 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.478241    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:51:01.579300 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.579267    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad82845c-be52-4b9a-b04c-84f7ae6127f1-proxy-tls\") pod \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") "
Sep 17 15:51:01.579489 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.579357    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad82845c-be52-4b9a-b04c-84f7ae6127f1-openshift-service-ca-bundle\") pod \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\" (UID: \"ad82845c-be52-4b9a-b04c-84f7ae6127f1\") "
Sep 17 15:51:01.579716 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.579689    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad82845c-be52-4b9a-b04c-84f7ae6127f1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ad82845c-be52-4b9a-b04c-84f7ae6127f1" (UID: "ad82845c-be52-4b9a-b04c-84f7ae6127f1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:51:01.581332 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.581313    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad82845c-be52-4b9a-b04c-84f7ae6127f1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ad82845c-be52-4b9a-b04c-84f7ae6127f1" (UID: "ad82845c-be52-4b9a-b04c-84f7ae6127f1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:51:01.680610 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.680571    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad82845c-be52-4b9a-b04c-84f7ae6127f1-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:51:01.680610 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.680603    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad82845c-be52-4b9a-b04c-84f7ae6127f1-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:51:01.872788 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.872749    2568 generic.go:356] "Generic (PLEG): container finished" podID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerID="2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4" exitCode=137
Sep 17 15:51:01.872964 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.872794    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"
Sep 17 15:51:01.872964 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.872832    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" event={"ID":"ad82845c-be52-4b9a-b04c-84f7ae6127f1","Type":"ContainerDied","Data":"2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4"}
Sep 17 15:51:01.872964 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.872908    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d" event={"ID":"ad82845c-be52-4b9a-b04c-84f7ae6127f1","Type":"ContainerDied","Data":"4eddbb3987f1aa0a764516f69dbacf2fc60dc74c737f40e79370c282770ec7d0"}
Sep 17 15:51:01.872964 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.872931    2568 scope.go:117] "RemoveContainer" containerID="2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4"
Sep 17 15:51:01.881440 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.881421    2568 scope.go:117] "RemoveContainer" containerID="2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4"
Sep 17 15:51:01.881718 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:01.881700    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4\": container with ID starting with 2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4 not found: ID does not exist" containerID="2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4"
Sep 17 15:51:01.881769 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.881727    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4"} err="failed to get container status \"2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4\": rpc error: code = NotFound desc = could not find container \"2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4\": container with ID starting with 2181a49682cecc9243286612c1eb9fb4c648a85bf9efc2cb4a7ece6ac3de6db4 not found: ID does not exist"
Sep 17 15:51:01.897951 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.897925    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"]
Sep 17 15:51:01.906062 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:01.906041    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-13ed5-64c7cc7cf4-hkn7d"]
Sep 17 15:51:03.830609 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:03.830569    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" path="/var/lib/kubelet/pods/ad82845c-be52-4b9a-b04c-84f7ae6127f1/volumes"
Sep 17 15:51:09.753958 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:09.753864    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 17 15:51:17.193588 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.193550    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"]
Sep 17 15:51:17.194000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.193913    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5"
Sep 17 15:51:17.194000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.193925    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5"
Sep 17 15:51:17.194000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.193944    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container"
Sep 17 15:51:17.194000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.193949    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container"
Sep 17 15:51:17.194000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.193960    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kube-rbac-proxy"
Sep 17 15:51:17.194000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.193966    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kube-rbac-proxy"
Sep 17 15:51:17.194191 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.194017    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kube-rbac-proxy"
Sep 17 15:51:17.194191 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.194026    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="42548f06-0c80-4f40-bafa-0e3ecd7a0b83" containerName="kserve-container"
Sep 17 15:51:17.194191 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.194034    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad82845c-be52-4b9a-b04c-84f7ae6127f1" containerName="switch-graph-13ed5"
Sep 17 15:51:17.198299 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.198279    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:17.208077 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.208055    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Sep 17 15:51:17.208194 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.208133    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Sep 17 15:51:17.216449 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.216426    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls\") pod \"model-chainer-6489bb6cff-9j8kz\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") " pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:17.217178 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.216504    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338b15a8-45f8-4124-9ddb-19e096a10c6c-openshift-service-ca-bundle\") pod \"model-chainer-6489bb6cff-9j8kz\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") " pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:17.218369 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.218346    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"]
Sep 17 15:51:17.317222 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.317188    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls\") pod \"model-chainer-6489bb6cff-9j8kz\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") " pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:17.317400 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.317251    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338b15a8-45f8-4124-9ddb-19e096a10c6c-openshift-service-ca-bundle\") pod \"model-chainer-6489bb6cff-9j8kz\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") " pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:17.317400 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:17.317342    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found
Sep 17 15:51:17.317473 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:17.317423    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls podName:338b15a8-45f8-4124-9ddb-19e096a10c6c nodeName:}" failed. No retries permitted until 2026-09-17 15:51:17.817402204 +0000 UTC m=+722.405446788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls") pod "model-chainer-6489bb6cff-9j8kz" (UID: "338b15a8-45f8-4124-9ddb-19e096a10c6c") : secret "model-chainer-serving-cert" not found
Sep 17 15:51:17.317886 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.317834    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338b15a8-45f8-4124-9ddb-19e096a10c6c-openshift-service-ca-bundle\") pod \"model-chainer-6489bb6cff-9j8kz\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") " pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:17.820649 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.820600    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls\") pod \"model-chainer-6489bb6cff-9j8kz\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") " pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:17.823042 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:17.823018    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls\") pod \"model-chainer-6489bb6cff-9j8kz\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") " pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:18.107660 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:18.107581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:18.380234 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:18.380206    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"]
Sep 17 15:51:18.381731 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:51:18.381702    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338b15a8_45f8_4124_9ddb_19e096a10c6c.slice/crio-b027bb4dab74ef8441b454ffecfeda3eca4d27d770db7d31ac9cd6e0c8603b87 WatchSource:0}: Error finding container b027bb4dab74ef8441b454ffecfeda3eca4d27d770db7d31ac9cd6e0c8603b87: Status 404 returned error can't find the container with id b027bb4dab74ef8441b454ffecfeda3eca4d27d770db7d31ac9cd6e0c8603b87
Sep 17 15:51:18.951133 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:18.951095    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" event={"ID":"338b15a8-45f8-4124-9ddb-19e096a10c6c","Type":"ContainerStarted","Data":"8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a"}
Sep 17 15:51:18.951342 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:18.951158    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:18.951342 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:18.951178    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" event={"ID":"338b15a8-45f8-4124-9ddb-19e096a10c6c","Type":"ContainerStarted","Data":"b027bb4dab74ef8441b454ffecfeda3eca4d27d770db7d31ac9cd6e0c8603b87"}
Sep 17 15:51:18.977147 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:18.977092    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podStartSLOduration=1.977077556 podStartE2EDuration="1.977077556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:51:18.976203964 +0000 UTC m=+723.564248552" watchObservedRunningTime="2026-09-17 15:51:18.977077556 +0000 UTC m=+723.565122182"
Sep 17 15:51:19.754275 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:19.754246    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:51:24.960969 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:24.960936    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:27.682686 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:27.682654    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"]
Sep 17 15:51:27.683131 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:27.682852    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer" containerID="cri-o://8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a" gracePeriod=30
Sep 17 15:51:28.013656 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.013624    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"]
Sep 17 15:51:28.017268 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.017249    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.020132 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.020112    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9ed13-kube-rbac-proxy-sar-config\""
Sep 17 15:51:28.020372 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.020355    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9ed13-predictor-serving-cert\""
Sep 17 15:51:28.029939 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.029914    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"]
Sep 17 15:51:28.107900 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.107867    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kcvn\" (UniqueName: \"kubernetes.io/projected/129fb228-a14e-428c-9025-f6947f62394f-kube-api-access-5kcvn\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.108057 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.107943    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/129fb228-a14e-428c-9025-f6947f62394f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.108057 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.107984    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.209206 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.209165    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/129fb228-a14e-428c-9025-f6947f62394f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.209390 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.209223    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.209390 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.209275    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kcvn\" (UniqueName: \"kubernetes.io/projected/129fb228-a14e-428c-9025-f6947f62394f-kube-api-access-5kcvn\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.209530 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:28.209392    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-serving-cert: secret "success-200-isvc-9ed13-predictor-serving-cert" not found
Sep 17 15:51:28.209530 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:28.209493    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls podName:129fb228-a14e-428c-9025-f6947f62394f nodeName:}" failed. No retries permitted until 2026-09-17 15:51:28.709471304 +0000 UTC m=+733.297515886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls") pod "success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" (UID: "129fb228-a14e-428c-9025-f6947f62394f") : secret "success-200-isvc-9ed13-predictor-serving-cert" not found
Sep 17 15:51:28.209930 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.209913    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/129fb228-a14e-428c-9025-f6947f62394f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.226180 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.226155    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kcvn\" (UniqueName: \"kubernetes.io/projected/129fb228-a14e-428c-9025-f6947f62394f-kube-api-access-5kcvn\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.713811 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.713767    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.716434 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.716409    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls\") pod \"success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") " pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:28.927712 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:28.927672    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:29.212140 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:29.212113    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"]
Sep 17 15:51:29.214930 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:51:29.214891    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod129fb228_a14e_428c_9025_f6947f62394f.slice/crio-ab30f1d177d5e7a75488f8b5d4bc24200a82275f2eb48639a27ffc8406f9ae48 WatchSource:0}: Error finding container ab30f1d177d5e7a75488f8b5d4bc24200a82275f2eb48639a27ffc8406f9ae48: Status 404 returned error can't find the container with id ab30f1d177d5e7a75488f8b5d4bc24200a82275f2eb48639a27ffc8406f9ae48
Sep 17 15:51:29.960083 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:29.960037    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:51:30.003032 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:30.002995    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" event={"ID":"129fb228-a14e-428c-9025-f6947f62394f","Type":"ContainerStarted","Data":"8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575"}
Sep 17 15:51:30.003236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:30.003066    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:30.003236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:30.003095    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:30.003236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:30.003118    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" event={"ID":"129fb228-a14e-428c-9025-f6947f62394f","Type":"ContainerStarted","Data":"8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08"}
Sep 17 15:51:30.003236 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:30.003148    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" event={"ID":"129fb228-a14e-428c-9025-f6947f62394f","Type":"ContainerStarted","Data":"ab30f1d177d5e7a75488f8b5d4bc24200a82275f2eb48639a27ffc8406f9ae48"}
Sep 17 15:51:30.004491 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:30.004459    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 17 15:51:30.032135 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:30.032091    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podStartSLOduration=3.032075645 podStartE2EDuration="3.032075645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:51:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:51:30.0310323 +0000 UTC m=+734.619076887" watchObservedRunningTime="2026-09-17 15:51:30.032075645 +0000 UTC m=+734.620120306"
Sep 17 15:51:31.008404 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.008363    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 17 15:51:31.608156 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.608124    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"]
Sep 17 15:51:31.611629 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.611608    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:31.615707 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.615690    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-795b5-serving-cert\""
Sep 17 15:51:31.615866 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.615847    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-795b5-kube-rbac-proxy-sar-config\""
Sep 17 15:51:31.633978 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.633942    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"]
Sep 17 15:51:31.642521 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.642494    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc695e7e-6dbf-48cd-a05a-8df3044ac674-openshift-service-ca-bundle\") pod \"switch-graph-795b5-5b65f768bf-hkwd2\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") " pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:31.642639 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.642589    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls\") pod \"switch-graph-795b5-5b65f768bf-hkwd2\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") " pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:31.743077 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.743043    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc695e7e-6dbf-48cd-a05a-8df3044ac674-openshift-service-ca-bundle\") pod \"switch-graph-795b5-5b65f768bf-hkwd2\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") " pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:31.743274 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.743216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls\") pod \"switch-graph-795b5-5b65f768bf-hkwd2\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") " pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:31.743347 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:31.743329    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-795b5-serving-cert: secret "switch-graph-795b5-serving-cert" not found
Sep 17 15:51:31.743404 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:31.743384    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls podName:cc695e7e-6dbf-48cd-a05a-8df3044ac674 nodeName:}" failed. No retries permitted until 2026-09-17 15:51:32.243368635 +0000 UTC m=+736.831413199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls") pod "switch-graph-795b5-5b65f768bf-hkwd2" (UID: "cc695e7e-6dbf-48cd-a05a-8df3044ac674") : secret "switch-graph-795b5-serving-cert" not found
Sep 17 15:51:31.743753 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:31.743730    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc695e7e-6dbf-48cd-a05a-8df3044ac674-openshift-service-ca-bundle\") pod \"switch-graph-795b5-5b65f768bf-hkwd2\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") " pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:32.247697 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:32.247666    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls\") pod \"switch-graph-795b5-5b65f768bf-hkwd2\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") " pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:32.249993 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:32.249972    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls\") pod \"switch-graph-795b5-5b65f768bf-hkwd2\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") " pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:32.522140 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:32.522050    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:32.810139 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:32.810063    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"]
Sep 17 15:51:32.815065 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:51:32.815026    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc695e7e_6dbf_48cd_a05a_8df3044ac674.slice/crio-75ef4b22f892781e5148be18eee13f271e28a345413117a4e37a7a39a16f12be WatchSource:0}: Error finding container 75ef4b22f892781e5148be18eee13f271e28a345413117a4e37a7a39a16f12be: Status 404 returned error can't find the container with id 75ef4b22f892781e5148be18eee13f271e28a345413117a4e37a7a39a16f12be
Sep 17 15:51:33.018886 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:33.018832    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" event={"ID":"cc695e7e-6dbf-48cd-a05a-8df3044ac674","Type":"ContainerStarted","Data":"99c82f1f6b48ecc009c106041a445f8a0799e5639d6943928e807da99cfd1e3d"}
Sep 17 15:51:33.019057 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:33.018906    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" event={"ID":"cc695e7e-6dbf-48cd-a05a-8df3044ac674","Type":"ContainerStarted","Data":"75ef4b22f892781e5148be18eee13f271e28a345413117a4e37a7a39a16f12be"}
Sep 17 15:51:33.019057 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:33.018945    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:34.959568 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:34.959528    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:51:36.012915 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:36.012885    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:51:36.013317 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:36.013280    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 17 15:51:36.037979 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:36.037931    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podStartSLOduration=5.037916145 podStartE2EDuration="5.037916145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:51:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:51:33.06624319 +0000 UTC m=+737.654287776" watchObservedRunningTime="2026-09-17 15:51:36.037916145 +0000 UTC m=+740.625960732"
Sep 17 15:51:39.035618 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:39.035584    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 15:51:39.959715 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:39.959674    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:51:39.959920 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:39.959777    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:44.959635 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:44.959593    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:51:46.014147 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:46.014106    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 17 15:51:49.960417 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:49.960372    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:51:54.960118 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:54.960065    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:51:56.014208 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:56.014159    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 17 15:51:57.835396 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:57.835368    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:57.973948 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:57.973919    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338b15a8-45f8-4124-9ddb-19e096a10c6c-openshift-service-ca-bundle\") pod \"338b15a8-45f8-4124-9ddb-19e096a10c6c\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") "
Sep 17 15:51:57.974127 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:57.974053    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls\") pod \"338b15a8-45f8-4124-9ddb-19e096a10c6c\" (UID: \"338b15a8-45f8-4124-9ddb-19e096a10c6c\") "
Sep 17 15:51:57.974300 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:57.974277    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338b15a8-45f8-4124-9ddb-19e096a10c6c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "338b15a8-45f8-4124-9ddb-19e096a10c6c" (UID: "338b15a8-45f8-4124-9ddb-19e096a10c6c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:51:57.976180 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:57.976148    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "338b15a8-45f8-4124-9ddb-19e096a10c6c" (UID: "338b15a8-45f8-4124-9ddb-19e096a10c6c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:51:58.075091 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.075046    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/338b15a8-45f8-4124-9ddb-19e096a10c6c-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:51:58.075091 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.075082    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338b15a8-45f8-4124-9ddb-19e096a10c6c-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:51:58.158659 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.158624    2568 generic.go:356] "Generic (PLEG): container finished" podID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerID="8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a" exitCode=0
Sep 17 15:51:58.158659 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.158666    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"
Sep 17 15:51:58.158901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.158710    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" event={"ID":"338b15a8-45f8-4124-9ddb-19e096a10c6c","Type":"ContainerDied","Data":"8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a"}
Sep 17 15:51:58.158901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.158760    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz" event={"ID":"338b15a8-45f8-4124-9ddb-19e096a10c6c","Type":"ContainerDied","Data":"b027bb4dab74ef8441b454ffecfeda3eca4d27d770db7d31ac9cd6e0c8603b87"}
Sep 17 15:51:58.158901 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.158783    2568 scope.go:117] "RemoveContainer" containerID="8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a"
Sep 17 15:51:58.167484 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.167467    2568 scope.go:117] "RemoveContainer" containerID="8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a"
Sep 17 15:51:58.167778 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:51:58.167756    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a\": container with ID starting with 8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a not found: ID does not exist" containerID="8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a"
Sep 17 15:51:58.167825 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.167788    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a"} err="failed to get container status \"8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a\": rpc error: code = NotFound desc = could not find container \"8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a\": container with ID starting with 8d11fae2f148fcfcc940c13a97a5ef764f8079921747f1d4f0a03ef34d4c5d8a not found: ID does not exist"
Sep 17 15:51:58.184940 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.184913    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"]
Sep 17 15:51:58.192053 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:58.192029    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6489bb6cff-9j8kz"]
Sep 17 15:51:59.831705 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:51:59.831670    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" path="/var/lib/kubelet/pods/338b15a8-45f8-4124-9ddb-19e096a10c6c/volumes"
Sep 17 15:52:06.014203 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:06.014165    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 17 15:52:16.013992 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:16.013954    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 15:52:27.923595 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.923543    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"]
Sep 17 15:52:27.924160 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.924140    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer"
Sep 17 15:52:27.924333 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.924162    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer"
Sep 17 15:52:27.924333 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.924265    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="338b15a8-45f8-4124-9ddb-19e096a10c6c" containerName="model-chainer"
Sep 17 15:52:27.928573 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.928550    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:27.931314 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.931288    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9ed13-kube-rbac-proxy-sar-config\""
Sep 17 15:52:27.931421 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.931297    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9ed13-serving-cert\""
Sep 17 15:52:27.937551 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:27.937501    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"]
Sep 17 15:52:28.033864 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.033810    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls\") pod \"sequence-graph-9ed13-5b8cd85464-qsffb\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") " pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:28.034036 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.033976    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/661f10dc-ec3d-4353-b855-f87f4a06b1f6-openshift-service-ca-bundle\") pod \"sequence-graph-9ed13-5b8cd85464-qsffb\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") " pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:28.135218 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.135175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls\") pod \"sequence-graph-9ed13-5b8cd85464-qsffb\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") " pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:28.135391 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.135252    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/661f10dc-ec3d-4353-b855-f87f4a06b1f6-openshift-service-ca-bundle\") pod \"sequence-graph-9ed13-5b8cd85464-qsffb\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") " pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:28.135391 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:52:28.135328    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-9ed13-serving-cert: secret "sequence-graph-9ed13-serving-cert" not found
Sep 17 15:52:28.135470 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:52:28.135407    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls podName:661f10dc-ec3d-4353-b855-f87f4a06b1f6 nodeName:}" failed. No retries permitted until 2026-09-17 15:52:28.635389363 +0000 UTC m=+793.223433927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls") pod "sequence-graph-9ed13-5b8cd85464-qsffb" (UID: "661f10dc-ec3d-4353-b855-f87f4a06b1f6") : secret "sequence-graph-9ed13-serving-cert" not found
Sep 17 15:52:28.135866 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.135822    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/661f10dc-ec3d-4353-b855-f87f4a06b1f6-openshift-service-ca-bundle\") pod \"sequence-graph-9ed13-5b8cd85464-qsffb\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") " pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:28.640400 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.640353    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls\") pod \"sequence-graph-9ed13-5b8cd85464-qsffb\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") " pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:28.642775 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.642742    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls\") pod \"sequence-graph-9ed13-5b8cd85464-qsffb\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") " pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:28.839799 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:28.839754    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:29.112808 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:29.112783    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"]
Sep 17 15:52:29.114879 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:52:29.114851    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod661f10dc_ec3d_4353_b855_f87f4a06b1f6.slice/crio-314fc7a696a8b79662121b4e3cd6d8747a9a198b2aa456ceeac98a86c22eeccb WatchSource:0}: Error finding container 314fc7a696a8b79662121b4e3cd6d8747a9a198b2aa456ceeac98a86c22eeccb: Status 404 returned error can't find the container with id 314fc7a696a8b79662121b4e3cd6d8747a9a198b2aa456ceeac98a86c22eeccb
Sep 17 15:52:29.305596 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:29.305565    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" event={"ID":"661f10dc-ec3d-4353-b855-f87f4a06b1f6","Type":"ContainerStarted","Data":"6e248acc80a4c2bdef65aa964ffff9c9909eb578538de56c4a5a6b258cedfbfd"}
Sep 17 15:52:29.305812 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:29.305621    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" event={"ID":"661f10dc-ec3d-4353-b855-f87f4a06b1f6","Type":"ContainerStarted","Data":"314fc7a696a8b79662121b4e3cd6d8747a9a198b2aa456ceeac98a86c22eeccb"}
Sep 17 15:52:29.305812 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:29.305670    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:52:29.331226 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:29.331177    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podStartSLOduration=2.331160295 podStartE2EDuration="2.331160295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:52:29.328167492 +0000 UTC m=+793.916212082" watchObservedRunningTime="2026-09-17 15:52:29.331160295 +0000 UTC m=+793.919204882"
Sep 17 15:52:35.317268 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:52:35.317229    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 15:54:15.819387 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:54:15.819355    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:54:15.820625 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:54:15.820599    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:59:15.849180 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:15.849105    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:59:15.851442 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:15.851416    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 15:59:46.309991 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.309959    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"]
Sep 17 15:59:46.310407 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.310145    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5" containerID="cri-o://99c82f1f6b48ecc009c106041a445f8a0799e5639d6943928e807da99cfd1e3d" gracePeriod=30
Sep 17 15:59:46.457330 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.457296    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"]
Sep 17 15:59:46.457555 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.457512    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container" containerID="cri-o://31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb" gracePeriod=30
Sep 17 15:59:46.457628 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.457544    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kube-rbac-proxy" containerID="cri-o://470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610" gracePeriod=30
Sep 17 15:59:46.607659 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.607624    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"]
Sep 17 15:59:46.611392 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.611370    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.613995 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.613971    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8d4df-predictor-serving-cert\""
Sep 17 15:59:46.614108 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.614018    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8d4df-kube-rbac-proxy-sar-config\""
Sep 17 15:59:46.631610 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.631583    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"]
Sep 17 15:59:46.755000 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.754957    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.755164 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.755038    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2crdg\" (UniqueName: \"kubernetes.io/projected/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-api-access-2crdg\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.755164 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.755100    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d22eccc7-996a-4b17-9565-57d8016c2f6a-proxy-tls\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.856328 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.856249    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d22eccc7-996a-4b17-9565-57d8016c2f6a-proxy-tls\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.856477 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.856337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.856477 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.856357    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crdg\" (UniqueName: \"kubernetes.io/projected/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-api-access-2crdg\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.857036 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.857002    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.858661 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.858637    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d22eccc7-996a-4b17-9565-57d8016c2f6a-proxy-tls\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.869086 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.869060    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2crdg\" (UniqueName: \"kubernetes.io/projected/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-api-access-2crdg\") pod \"success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") " pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:46.922673 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:46.922642    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:47.213803 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:47.213773    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"]
Sep 17 15:59:47.216469 ip-10-0-140-183 kubenswrapper[2568]: W0917 15:59:47.216441    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22eccc7_996a_4b17_9565_57d8016c2f6a.slice/crio-29ce000056aede99ff9104fb06fef1bde362315297f420595c29f8226bcb0b40 WatchSource:0}: Error finding container 29ce000056aede99ff9104fb06fef1bde362315297f420595c29f8226bcb0b40: Status 404 returned error can't find the container with id 29ce000056aede99ff9104fb06fef1bde362315297f420595c29f8226bcb0b40
Sep 17 15:59:47.302815 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:47.302788    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" event={"ID":"d22eccc7-996a-4b17-9565-57d8016c2f6a","Type":"ContainerStarted","Data":"0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c"}
Sep 17 15:59:47.302918 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:47.302900    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" event={"ID":"d22eccc7-996a-4b17-9565-57d8016c2f6a","Type":"ContainerStarted","Data":"29ce000056aede99ff9104fb06fef1bde362315297f420595c29f8226bcb0b40"}
Sep 17 15:59:47.304539 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:47.304519    2568 generic.go:356] "Generic (PLEG): container finished" podID="d8039d28-570c-40c4-bb53-0d479af97b96" containerID="470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610" exitCode=2
Sep 17 15:59:47.304699 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:47.304570    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" event={"ID":"d8039d28-570c-40c4-bb53-0d479af97b96","Type":"ContainerDied","Data":"470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610"}
Sep 17 15:59:48.310382 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:48.310348    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" event={"ID":"d22eccc7-996a-4b17-9565-57d8016c2f6a","Type":"ContainerStarted","Data":"9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d"}
Sep 17 15:59:48.310799 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:48.310418    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:48.310799 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:48.310442    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:48.311913 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:48.311886    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 17 15:59:48.335405 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:48.335353    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podStartSLOduration=2.335334073 podStartE2EDuration="2.335334073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:59:48.333766676 +0000 UTC m=+1232.921811265" watchObservedRunningTime="2026-09-17 15:59:48.335334073 +0000 UTC m=+1232.923378661"
Sep 17 15:59:49.033933 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.033889    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:59:49.315992 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.315895    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 17 15:59:49.716911 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.716889    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:59:49.885362 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.885267    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8039d28-570c-40c4-bb53-0d479af97b96-proxy-tls\") pod \"d8039d28-570c-40c4-bb53-0d479af97b96\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") "
Sep 17 15:59:49.885362 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.885334    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z88x5\" (UniqueName: \"kubernetes.io/projected/d8039d28-570c-40c4-bb53-0d479af97b96-kube-api-access-z88x5\") pod \"d8039d28-570c-40c4-bb53-0d479af97b96\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") "
Sep 17 15:59:49.885618 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.885456    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8039d28-570c-40c4-bb53-0d479af97b96-kube-rbac-proxy-sar-config\") pod \"d8039d28-570c-40c4-bb53-0d479af97b96\" (UID: \"d8039d28-570c-40c4-bb53-0d479af97b96\") "
Sep 17 15:59:49.885770 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.885743    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8039d28-570c-40c4-bb53-0d479af97b96-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d8039d28-570c-40c4-bb53-0d479af97b96" (UID: "d8039d28-570c-40c4-bb53-0d479af97b96"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:59:49.887455 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.887432    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8039d28-570c-40c4-bb53-0d479af97b96-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d8039d28-570c-40c4-bb53-0d479af97b96" (UID: "d8039d28-570c-40c4-bb53-0d479af97b96"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:59:49.887583 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.887489    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8039d28-570c-40c4-bb53-0d479af97b96-kube-api-access-z88x5" (OuterVolumeSpecName: "kube-api-access-z88x5") pod "d8039d28-570c-40c4-bb53-0d479af97b96" (UID: "d8039d28-570c-40c4-bb53-0d479af97b96"). InnerVolumeSpecName "kube-api-access-z88x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:59:49.986918 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.986875    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8039d28-570c-40c4-bb53-0d479af97b96-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:59:49.986918 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.986913    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z88x5\" (UniqueName: \"kubernetes.io/projected/d8039d28-570c-40c4-bb53-0d479af97b96-kube-api-access-z88x5\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:59:49.986918 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:49.986926    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8039d28-570c-40c4-bb53-0d479af97b96-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 15:59:50.322025 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.321991    2568 generic.go:356] "Generic (PLEG): container finished" podID="d8039d28-570c-40c4-bb53-0d479af97b96" containerID="31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb" exitCode=0
Sep 17 15:59:50.322438 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.322046    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"
Sep 17 15:59:50.322438 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.322077    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" event={"ID":"d8039d28-570c-40c4-bb53-0d479af97b96","Type":"ContainerDied","Data":"31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb"}
Sep 17 15:59:50.322438 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.322146    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns" event={"ID":"d8039d28-570c-40c4-bb53-0d479af97b96","Type":"ContainerDied","Data":"4575c1a8175f4161633da86d403deaa9413c563d036e8d8f4992751290d56c32"}
Sep 17 15:59:50.322438 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.322184    2568 scope.go:117] "RemoveContainer" containerID="470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610"
Sep 17 15:59:50.331190 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.331170    2568 scope.go:117] "RemoveContainer" containerID="31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb"
Sep 17 15:59:50.339515 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.339493    2568 scope.go:117] "RemoveContainer" containerID="470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610"
Sep 17 15:59:50.339899 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:59:50.339819    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610\": container with ID starting with 470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610 not found: ID does not exist" containerID="470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610"
Sep 17 15:59:50.339984 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.339899    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610"} err="failed to get container status \"470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610\": rpc error: code = NotFound desc = could not find container \"470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610\": container with ID starting with 470e0a09e62b531852f7c04f81b67119bdde9df308a9440c019ad83519ef7610 not found: ID does not exist"
Sep 17 15:59:50.339984 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.339925    2568 scope.go:117] "RemoveContainer" containerID="31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb"
Sep 17 15:59:50.340197 ip-10-0-140-183 kubenswrapper[2568]: E0917 15:59:50.340181    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb\": container with ID starting with 31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb not found: ID does not exist" containerID="31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb"
Sep 17 15:59:50.340248 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.340203    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb"} err="failed to get container status \"31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb\": rpc error: code = NotFound desc = could not find container \"31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb\": container with ID starting with 31c4d820d8e3e7969e24afd9cabdd8ce76e475f3a46328f58336f927a08b5ecb not found: ID does not exist"
Sep 17 15:59:50.350502 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.350478    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"]
Sep 17 15:59:50.354074 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:50.354048    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-795b5-predictor-76d9994664-t6lns"]
Sep 17 15:59:51.832259 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:51.832219    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" path="/var/lib/kubelet/pods/d8039d28-570c-40c4-bb53-0d479af97b96/volumes"
Sep 17 15:59:54.033306 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:54.033261    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:59:54.320710 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:54.320629    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 15:59:54.321078 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:54.321041    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 17 15:59:59.033730 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:59.033685    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:59:59.034169 ip-10-0-140-183 kubenswrapper[2568]: I0917 15:59:59.033798    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 16:00:04.033516 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:04.033471    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:00:04.321963 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:04.321877    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 17 16:00:09.033605 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:09.033502    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:00:14.033505 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:14.033456    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:00:14.321889 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:14.321790    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 17 16:00:16.446954 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.446925    2568 generic.go:356] "Generic (PLEG): container finished" podID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerID="99c82f1f6b48ecc009c106041a445f8a0799e5639d6943928e807da99cfd1e3d" exitCode=0
Sep 17 16:00:16.447274 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.446964    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" event={"ID":"cc695e7e-6dbf-48cd-a05a-8df3044ac674","Type":"ContainerDied","Data":"99c82f1f6b48ecc009c106041a445f8a0799e5639d6943928e807da99cfd1e3d"}
Sep 17 16:00:16.447274 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.446999    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2" event={"ID":"cc695e7e-6dbf-48cd-a05a-8df3044ac674","Type":"ContainerDied","Data":"75ef4b22f892781e5148be18eee13f271e28a345413117a4e37a7a39a16f12be"}
Sep 17 16:00:16.447274 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.447013    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75ef4b22f892781e5148be18eee13f271e28a345413117a4e37a7a39a16f12be"
Sep 17 16:00:16.459113 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.459092    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 16:00:16.511706 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.511680    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc695e7e-6dbf-48cd-a05a-8df3044ac674-openshift-service-ca-bundle\") pod \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") "
Sep 17 16:00:16.511889 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.511722    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls\") pod \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\" (UID: \"cc695e7e-6dbf-48cd-a05a-8df3044ac674\") "
Sep 17 16:00:16.512011 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.511988    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc695e7e-6dbf-48cd-a05a-8df3044ac674-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cc695e7e-6dbf-48cd-a05a-8df3044ac674" (UID: "cc695e7e-6dbf-48cd-a05a-8df3044ac674"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:00:16.513757 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.513725    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cc695e7e-6dbf-48cd-a05a-8df3044ac674" (UID: "cc695e7e-6dbf-48cd-a05a-8df3044ac674"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:00:16.613232 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.613147    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc695e7e-6dbf-48cd-a05a-8df3044ac674-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:00:16.613232 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:16.613178    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc695e7e-6dbf-48cd-a05a-8df3044ac674-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:00:17.451234 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:17.451201    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"
Sep 17 16:00:17.475054 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:17.475024    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"]
Sep 17 16:00:17.483076 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:17.483052    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-795b5-5b65f768bf-hkwd2"]
Sep 17 16:00:17.830869 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:17.830775    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" path="/var/lib/kubelet/pods/cc695e7e-6dbf-48cd-a05a-8df3044ac674/volumes"
Sep 17 16:00:24.321131 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:24.321089    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Sep 17 16:00:34.321810 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:34.321778    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 16:00:42.662032 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:42.661992    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"]
Sep 17 16:00:42.662483 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:42.662237    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13" containerID="cri-o://6e248acc80a4c2bdef65aa964ffff9c9909eb578538de56c4a5a6b258cedfbfd" gracePeriod=30
Sep 17 16:00:42.868637 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:42.868605    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"]
Sep 17 16:00:42.868870 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:42.868830    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container" containerID="cri-o://8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08" gracePeriod=30
Sep 17 16:00:42.868966 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:42.868889    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kube-rbac-proxy" containerID="cri-o://8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575" gracePeriod=30
Sep 17 16:00:43.022909 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.022874    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"]
Sep 17 16:00:43.023247 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023234    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kube-rbac-proxy"
Sep 17 16:00:43.023291 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023249    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kube-rbac-proxy"
Sep 17 16:00:43.023291 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023264    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5"
Sep 17 16:00:43.023291 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023269    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5"
Sep 17 16:00:43.023291 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023277    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container"
Sep 17 16:00:43.023291 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023283    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container"
Sep 17 16:00:43.023452 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023343    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc695e7e-6dbf-48cd-a05a-8df3044ac674" containerName="switch-graph-795b5"
Sep 17 16:00:43.023452 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023351    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kube-rbac-proxy"
Sep 17 16:00:43.023452 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.023359    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8039d28-570c-40c4-bb53-0d479af97b96" containerName="kserve-container"
Sep 17 16:00:43.026568 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.026548    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.029215 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.029195    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-efa50-kube-rbac-proxy-sar-config\""
Sep 17 16:00:43.029314 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.029234    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-efa50-predictor-serving-cert\""
Sep 17 16:00:43.047850 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.047820    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"]
Sep 17 16:00:43.146907 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.146872    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d7d5e861-fd6b-4e7e-b632-4531528ebf56-proxy-tls\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.147079 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.146933    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbd98\" (UniqueName: \"kubernetes.io/projected/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-api-access-tbd98\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.147079 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.146962    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.248063 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.247979    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d7d5e861-fd6b-4e7e-b632-4531528ebf56-proxy-tls\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.248063 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.248029    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbd98\" (UniqueName: \"kubernetes.io/projected/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-api-access-tbd98\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.248063 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.248048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.248791 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.248764    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.250534 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.250503    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d7d5e861-fd6b-4e7e-b632-4531528ebf56-proxy-tls\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.262760 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.262738    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbd98\" (UniqueName: \"kubernetes.io/projected/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-api-access-tbd98\") pod \"success-200-isvc-efa50-predictor-5f56d967f4-2f9vv\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") " pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.336793 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.336761    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:43.573596 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.573511    2568 generic.go:356] "Generic (PLEG): container finished" podID="129fb228-a14e-428c-9025-f6947f62394f" containerID="8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575" exitCode=2
Sep 17 16:00:43.573596 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.573560    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" event={"ID":"129fb228-a14e-428c-9025-f6947f62394f","Type":"ContainerDied","Data":"8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575"}
Sep 17 16:00:43.617581 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:43.617555    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"]
Sep 17 16:00:43.619432 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:00:43.619399    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d5e861_fd6b_4e7e_b632_4531528ebf56.slice/crio-03be0f99cc175b7c5c1df99db24b0af37804d59f1e1d2aa03cb223396297dec6 WatchSource:0}: Error finding container 03be0f99cc175b7c5c1df99db24b0af37804d59f1e1d2aa03cb223396297dec6: Status 404 returned error can't find the container with id 03be0f99cc175b7c5c1df99db24b0af37804d59f1e1d2aa03cb223396297dec6
Sep 17 16:00:44.579796 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:44.579759    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" event={"ID":"d7d5e861-fd6b-4e7e-b632-4531528ebf56","Type":"ContainerStarted","Data":"434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7"}
Sep 17 16:00:44.580178 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:44.579866    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" event={"ID":"d7d5e861-fd6b-4e7e-b632-4531528ebf56","Type":"ContainerStarted","Data":"8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba"}
Sep 17 16:00:44.580178 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:44.579898    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" event={"ID":"d7d5e861-fd6b-4e7e-b632-4531528ebf56","Type":"ContainerStarted","Data":"03be0f99cc175b7c5c1df99db24b0af37804d59f1e1d2aa03cb223396297dec6"}
Sep 17 16:00:44.580178 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:44.580048    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:44.580178 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:44.580105    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:44.581374 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:44.581348    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 17 16:00:44.607148 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:44.607104    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podStartSLOduration=1.607090063 podStartE2EDuration="1.607090063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:00:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:00:44.606030662 +0000 UTC m=+1289.194075271" watchObservedRunningTime="2026-09-17 16:00:44.607090063 +0000 UTC m=+1289.195134651"
Sep 17 16:00:45.316099 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:45.316063    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:00:45.584767 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:45.584681    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 17 16:00:45.925116 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:45.925087    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 16:00:46.073792 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.073763    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/129fb228-a14e-428c-9025-f6947f62394f-kube-rbac-proxy-sar-config\") pod \"129fb228-a14e-428c-9025-f6947f62394f\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") "
Sep 17 16:00:46.073963 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.073799    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls\") pod \"129fb228-a14e-428c-9025-f6947f62394f\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") "
Sep 17 16:00:46.073963 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.073898    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kcvn\" (UniqueName: \"kubernetes.io/projected/129fb228-a14e-428c-9025-f6947f62394f-kube-api-access-5kcvn\") pod \"129fb228-a14e-428c-9025-f6947f62394f\" (UID: \"129fb228-a14e-428c-9025-f6947f62394f\") "
Sep 17 16:00:46.074100 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.074074    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/129fb228-a14e-428c-9025-f6947f62394f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "129fb228-a14e-428c-9025-f6947f62394f" (UID: "129fb228-a14e-428c-9025-f6947f62394f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:00:46.075851 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.075818    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "129fb228-a14e-428c-9025-f6947f62394f" (UID: "129fb228-a14e-428c-9025-f6947f62394f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:00:46.076009 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.075989    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129fb228-a14e-428c-9025-f6947f62394f-kube-api-access-5kcvn" (OuterVolumeSpecName: "kube-api-access-5kcvn") pod "129fb228-a14e-428c-9025-f6947f62394f" (UID: "129fb228-a14e-428c-9025-f6947f62394f"). InnerVolumeSpecName "kube-api-access-5kcvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:00:46.175080 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.175007    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kcvn\" (UniqueName: \"kubernetes.io/projected/129fb228-a14e-428c-9025-f6947f62394f-kube-api-access-5kcvn\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:00:46.175080 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.175033    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/129fb228-a14e-428c-9025-f6947f62394f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:00:46.175080 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.175046    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/129fb228-a14e-428c-9025-f6947f62394f-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:00:46.541071 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.541036    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"]
Sep 17 16:00:46.541414 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.541401    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container"
Sep 17 16:00:46.541490 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.541415    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container"
Sep 17 16:00:46.541490 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.541440    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kube-rbac-proxy"
Sep 17 16:00:46.541490 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.541446    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kube-rbac-proxy"
Sep 17 16:00:46.541671 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.541506    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kserve-container"
Sep 17 16:00:46.541671 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.541518    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="129fb228-a14e-428c-9025-f6947f62394f" containerName="kube-rbac-proxy"
Sep 17 16:00:46.544563 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.544546    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:46.548976 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.548930    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-8d4df-serving-cert\""
Sep 17 16:00:46.549096 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.549074    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-8d4df-kube-rbac-proxy-sar-config\""
Sep 17 16:00:46.555996 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.555970    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"]
Sep 17 16:00:46.590257 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.590223    2568 generic.go:356] "Generic (PLEG): container finished" podID="129fb228-a14e-428c-9025-f6947f62394f" containerID="8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08" exitCode=0
Sep 17 16:00:46.590604 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.590275    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" event={"ID":"129fb228-a14e-428c-9025-f6947f62394f","Type":"ContainerDied","Data":"8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08"}
Sep 17 16:00:46.590604 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.590312    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl" event={"ID":"129fb228-a14e-428c-9025-f6947f62394f","Type":"ContainerDied","Data":"ab30f1d177d5e7a75488f8b5d4bc24200a82275f2eb48639a27ffc8406f9ae48"}
Sep 17 16:00:46.590604 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.590338    2568 scope.go:117] "RemoveContainer" containerID="8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575"
Sep 17 16:00:46.590604 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.590277    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"
Sep 17 16:00:46.599151 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.599135    2568 scope.go:117] "RemoveContainer" containerID="8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08"
Sep 17 16:00:46.606567 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.606551    2568 scope.go:117] "RemoveContainer" containerID="8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575"
Sep 17 16:00:46.606805 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:00:46.606785    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575\": container with ID starting with 8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575 not found: ID does not exist" containerID="8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575"
Sep 17 16:00:46.606871 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.606811    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575"} err="failed to get container status \"8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575\": rpc error: code = NotFound desc = could not find container \"8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575\": container with ID starting with 8765731dc588aee63db713ce4cbe3ad6decf1e5dc9b4f77642e5d17e4c22f575 not found: ID does not exist"
Sep 17 16:00:46.606871 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.606827    2568 scope.go:117] "RemoveContainer" containerID="8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08"
Sep 17 16:00:46.607070 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:00:46.607053    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08\": container with ID starting with 8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08 not found: ID does not exist" containerID="8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08"
Sep 17 16:00:46.607110 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.607077    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08"} err="failed to get container status \"8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08\": rpc error: code = NotFound desc = could not find container \"8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08\": container with ID starting with 8e204e04995cd585269a25848645dcfbb88528a8c8acca7ea12ba435c0b4ef08 not found: ID does not exist"
Sep 17 16:00:46.616490 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.616470    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"]
Sep 17 16:00:46.637065 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.637045    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9ed13-predictor-d9d56cd6d-rzjcl"]
Sep 17 16:00:46.678430 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.678407    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-openshift-service-ca-bundle\") pod \"ensemble-graph-8d4df-585ccf9548-ks5b2\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") " pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:46.678530 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.678514    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls\") pod \"ensemble-graph-8d4df-585ccf9548-ks5b2\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") " pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:46.779539 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.779511    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls\") pod \"ensemble-graph-8d4df-585ccf9548-ks5b2\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") " pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:46.779661 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.779553    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-openshift-service-ca-bundle\") pod \"ensemble-graph-8d4df-585ccf9548-ks5b2\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") " pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:46.779715 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:00:46.779657    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-8d4df-serving-cert: secret "ensemble-graph-8d4df-serving-cert" not found
Sep 17 16:00:46.779753 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:00:46.779714    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls podName:b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef nodeName:}" failed. No retries permitted until 2026-09-17 16:00:47.279700311 +0000 UTC m=+1291.867744876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls") pod "ensemble-graph-8d4df-585ccf9548-ks5b2" (UID: "b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef") : secret "ensemble-graph-8d4df-serving-cert" not found
Sep 17 16:00:46.780169 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:46.780152    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-openshift-service-ca-bundle\") pod \"ensemble-graph-8d4df-585ccf9548-ks5b2\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") " pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:47.285827 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:47.285793    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls\") pod \"ensemble-graph-8d4df-585ccf9548-ks5b2\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") " pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:47.288104 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:47.288077    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls\") pod \"ensemble-graph-8d4df-585ccf9548-ks5b2\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") " pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:47.455041 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:47.455010    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:47.733068 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:47.733038    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"]
Sep 17 16:00:47.734431 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:00:47.734401    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c0d7a7_1181_4f27_90e2_d1fa0b6dc9ef.slice/crio-02639f450f0104a61cca1be0b0b2c693c9a657d3063e52ed6ffb5a63ba54572e WatchSource:0}: Error finding container 02639f450f0104a61cca1be0b0b2c693c9a657d3063e52ed6ffb5a63ba54572e: Status 404 returned error can't find the container with id 02639f450f0104a61cca1be0b0b2c693c9a657d3063e52ed6ffb5a63ba54572e
Sep 17 16:00:47.831885 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:47.831854    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="129fb228-a14e-428c-9025-f6947f62394f" path="/var/lib/kubelet/pods/129fb228-a14e-428c-9025-f6947f62394f/volumes"
Sep 17 16:00:48.600908 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:48.600870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" event={"ID":"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef","Type":"ContainerStarted","Data":"557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d"}
Sep 17 16:00:48.601141 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:48.600951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" event={"ID":"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef","Type":"ContainerStarted","Data":"02639f450f0104a61cca1be0b0b2c693c9a657d3063e52ed6ffb5a63ba54572e"}
Sep 17 16:00:48.601141 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:48.600974    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:48.626510 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:48.626461    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podStartSLOduration=2.626446748 podStartE2EDuration="2.626446748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:00:48.625143737 +0000 UTC m=+1293.213188322" watchObservedRunningTime="2026-09-17 16:00:48.626446748 +0000 UTC m=+1293.214491368"
Sep 17 16:00:50.316211 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:50.316171    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:00:50.589494 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:50.589408    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:00:50.590210 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:50.590178    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 17 16:00:54.612047 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:54.612019    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:00:55.315547 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:55.315509    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:00:55.315712 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:55.315647    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 16:00:56.614201 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.614121    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"]
Sep 17 16:00:56.614635 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.614414    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df" containerID="cri-o://557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d" gracePeriod=30
Sep 17 16:00:56.722983 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.722948    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"]
Sep 17 16:00:56.723297 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.723242    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container" containerID="cri-o://0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c" gracePeriod=30
Sep 17 16:00:56.723595 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.723272    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kube-rbac-proxy" containerID="cri-o://9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d" gracePeriod=30
Sep 17 16:00:56.824704 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.824659    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"]
Sep 17 16:00:56.828322 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.828295    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.830971 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.830935    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-bfbea-predictor-serving-cert\""
Sep 17 16:00:56.831103 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.831084    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-bfbea-kube-rbac-proxy-sar-config\""
Sep 17 16:00:56.841748 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.841704    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"]
Sep 17 16:00:56.867979 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.867917    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a5a38-4c86-43c8-8327-55470532dafd-proxy-tls\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.868084 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.868001    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xrcm\" (UniqueName: \"kubernetes.io/projected/198a5a38-4c86-43c8-8327-55470532dafd-kube-api-access-7xrcm\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.868266 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.868239    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a5a38-4c86-43c8-8327-55470532dafd-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.969518 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.969484    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a5a38-4c86-43c8-8327-55470532dafd-proxy-tls\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.969698 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.969526    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xrcm\" (UniqueName: \"kubernetes.io/projected/198a5a38-4c86-43c8-8327-55470532dafd-kube-api-access-7xrcm\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.969698 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.969653    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a5a38-4c86-43c8-8327-55470532dafd-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.970825 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.970796    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a5a38-4c86-43c8-8327-55470532dafd-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.972143 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.972122    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a5a38-4c86-43c8-8327-55470532dafd-proxy-tls\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:56.982564 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:56.982539    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xrcm\" (UniqueName: \"kubernetes.io/projected/198a5a38-4c86-43c8-8327-55470532dafd-kube-api-access-7xrcm\") pod \"success-200-isvc-bfbea-predictor-db56bc584-klx7c\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") " pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:57.140034 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.139956    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:57.414161 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.414138    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"]
Sep 17 16:00:57.415751 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:00:57.415729    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198a5a38_4c86_43c8_8327_55470532dafd.slice/crio-d397660dfdd59f33d66b3a48f5c095c194150a38ccf03ba0ac949a34fc219e57 WatchSource:0}: Error finding container d397660dfdd59f33d66b3a48f5c095c194150a38ccf03ba0ac949a34fc219e57: Status 404 returned error can't find the container with id d397660dfdd59f33d66b3a48f5c095c194150a38ccf03ba0ac949a34fc219e57
Sep 17 16:00:57.649975 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.649941    2568 generic.go:356] "Generic (PLEG): container finished" podID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerID="9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d" exitCode=2
Sep 17 16:00:57.650396 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.650022    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" event={"ID":"d22eccc7-996a-4b17-9565-57d8016c2f6a","Type":"ContainerDied","Data":"9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d"}
Sep 17 16:00:57.654421 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.654393    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" event={"ID":"198a5a38-4c86-43c8-8327-55470532dafd","Type":"ContainerStarted","Data":"6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd"}
Sep 17 16:00:57.654544 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.654446    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" event={"ID":"198a5a38-4c86-43c8-8327-55470532dafd","Type":"ContainerStarted","Data":"a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5"}
Sep 17 16:00:57.654544 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.654474    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:57.654544 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.654493    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" event={"ID":"198a5a38-4c86-43c8-8327-55470532dafd","Type":"ContainerStarted","Data":"d397660dfdd59f33d66b3a48f5c095c194150a38ccf03ba0ac949a34fc219e57"}
Sep 17 16:00:57.678123 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:57.678030    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podStartSLOduration=1.678015012 podStartE2EDuration="1.678015012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:00:57.676100032 +0000 UTC m=+1302.264144630" watchObservedRunningTime="2026-09-17 16:00:57.678015012 +0000 UTC m=+1302.266059599"
Sep 17 16:00:58.659075 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:58.659040    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:00:58.660445 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:58.660417    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 17 16:00:59.316770 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.316730    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Sep 17 16:00:59.610003 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.609913    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:00:59.664218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.664175    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 17 16:00:59.976943 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.976918    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 16:00:59.996107 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.996084    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d22eccc7-996a-4b17-9565-57d8016c2f6a-proxy-tls\") pod \"d22eccc7-996a-4b17-9565-57d8016c2f6a\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") "
Sep 17 16:00:59.996181 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.996144    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-rbac-proxy-sar-config\") pod \"d22eccc7-996a-4b17-9565-57d8016c2f6a\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") "
Sep 17 16:00:59.996181 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.996164    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2crdg\" (UniqueName: \"kubernetes.io/projected/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-api-access-2crdg\") pod \"d22eccc7-996a-4b17-9565-57d8016c2f6a\" (UID: \"d22eccc7-996a-4b17-9565-57d8016c2f6a\") "
Sep 17 16:00:59.996510 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.996487    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d22eccc7-996a-4b17-9565-57d8016c2f6a" (UID: "d22eccc7-996a-4b17-9565-57d8016c2f6a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:00:59.998140 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.998116    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-api-access-2crdg" (OuterVolumeSpecName: "kube-api-access-2crdg") pod "d22eccc7-996a-4b17-9565-57d8016c2f6a" (UID: "d22eccc7-996a-4b17-9565-57d8016c2f6a"). InnerVolumeSpecName "kube-api-access-2crdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:00:59.998238 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:00:59.998195    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d22eccc7-996a-4b17-9565-57d8016c2f6a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d22eccc7-996a-4b17-9565-57d8016c2f6a" (UID: "d22eccc7-996a-4b17-9565-57d8016c2f6a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:01:00.097279 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.097245    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d22eccc7-996a-4b17-9565-57d8016c2f6a-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:00.097279 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.097274    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:00.097279 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.097284    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2crdg\" (UniqueName: \"kubernetes.io/projected/d22eccc7-996a-4b17-9565-57d8016c2f6a-kube-api-access-2crdg\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:00.315623 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.315582    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:00.590749 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.590658    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 17 16:01:00.670073 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.670038    2568 generic.go:356] "Generic (PLEG): container finished" podID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerID="0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c" exitCode=0
Sep 17 16:01:00.670535 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.670090    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"
Sep 17 16:01:00.670535 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.670118    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" event={"ID":"d22eccc7-996a-4b17-9565-57d8016c2f6a","Type":"ContainerDied","Data":"0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c"}
Sep 17 16:01:00.670535 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.670177    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w" event={"ID":"d22eccc7-996a-4b17-9565-57d8016c2f6a","Type":"ContainerDied","Data":"29ce000056aede99ff9104fb06fef1bde362315297f420595c29f8226bcb0b40"}
Sep 17 16:01:00.670535 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.670202    2568 scope.go:117] "RemoveContainer" containerID="9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d"
Sep 17 16:01:00.679576 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.679557    2568 scope.go:117] "RemoveContainer" containerID="0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c"
Sep 17 16:01:00.686833 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.686816    2568 scope.go:117] "RemoveContainer" containerID="9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d"
Sep 17 16:01:00.687073 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:01:00.687055    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d\": container with ID starting with 9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d not found: ID does not exist" containerID="9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d"
Sep 17 16:01:00.687127 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.687081    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d"} err="failed to get container status \"9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d\": rpc error: code = NotFound desc = could not find container \"9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d\": container with ID starting with 9a186940a1329688d8781bc9fa90d5cb69a7f1f534e05873ecd095ef3cb59c7d not found: ID does not exist"
Sep 17 16:01:00.687127 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.687097    2568 scope.go:117] "RemoveContainer" containerID="0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c"
Sep 17 16:01:00.687293 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:01:00.687277    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c\": container with ID starting with 0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c not found: ID does not exist" containerID="0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c"
Sep 17 16:01:00.687337 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.687301    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c"} err="failed to get container status \"0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c\": rpc error: code = NotFound desc = could not find container \"0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c\": container with ID starting with 0a6d918f5520e491d6b7775536952f5ca79c04990766f805fca4325df5c07e4c not found: ID does not exist"
Sep 17 16:01:00.695148 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.695127    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"]
Sep 17 16:01:00.700470 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:00.700449    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8d4df-predictor-6b6598bf4d-h8w5w"]
Sep 17 16:01:01.830775 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:01.830741    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" path="/var/lib/kubelet/pods/d22eccc7-996a-4b17-9565-57d8016c2f6a/volumes"
Sep 17 16:01:04.610489 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:04.610444    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:04.669391 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:04.669359    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:01:04.669875 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:04.669830    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 17 16:01:05.316172 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:05.316134    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:09.609998 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:09.609960    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:09.610361 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:09.610070    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:01:10.315967 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:10.315931    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:10.591007 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:10.590914    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 17 16:01:12.728888 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:12.728826    2568 generic.go:356] "Generic (PLEG): container finished" podID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerID="6e248acc80a4c2bdef65aa964ffff9c9909eb578538de56c4a5a6b258cedfbfd" exitCode=0
Sep 17 16:01:12.729240 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:12.728898    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" event={"ID":"661f10dc-ec3d-4353-b855-f87f4a06b1f6","Type":"ContainerDied","Data":"6e248acc80a4c2bdef65aa964ffff9c9909eb578538de56c4a5a6b258cedfbfd"}
Sep 17 16:01:12.815764 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:12.815737    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 16:01:12.907354 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:12.907324    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls\") pod \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") "
Sep 17 16:01:12.907515 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:12.907423    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/661f10dc-ec3d-4353-b855-f87f4a06b1f6-openshift-service-ca-bundle\") pod \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\" (UID: \"661f10dc-ec3d-4353-b855-f87f4a06b1f6\") "
Sep 17 16:01:12.907780 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:12.907758    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/661f10dc-ec3d-4353-b855-f87f4a06b1f6-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "661f10dc-ec3d-4353-b855-f87f4a06b1f6" (UID: "661f10dc-ec3d-4353-b855-f87f4a06b1f6"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:01:12.909430 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:12.909409    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "661f10dc-ec3d-4353-b855-f87f4a06b1f6" (UID: "661f10dc-ec3d-4353-b855-f87f4a06b1f6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:01:13.009118 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.009086    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/661f10dc-ec3d-4353-b855-f87f4a06b1f6-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:13.009118 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.009116    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/661f10dc-ec3d-4353-b855-f87f4a06b1f6-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:13.736743 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.736707    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb" event={"ID":"661f10dc-ec3d-4353-b855-f87f4a06b1f6","Type":"ContainerDied","Data":"314fc7a696a8b79662121b4e3cd6d8747a9a198b2aa456ceeac98a86c22eeccb"}
Sep 17 16:01:13.737197 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.736751    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"
Sep 17 16:01:13.737197 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.736803    2568 scope.go:117] "RemoveContainer" containerID="6e248acc80a4c2bdef65aa964ffff9c9909eb578538de56c4a5a6b258cedfbfd"
Sep 17 16:01:13.760446 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.760423    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"]
Sep 17 16:01:13.765000 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.764980    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9ed13-5b8cd85464-qsffb"]
Sep 17 16:01:13.830937 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:13.830912    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" path="/var/lib/kubelet/pods/661f10dc-ec3d-4353-b855-f87f4a06b1f6/volumes"
Sep 17 16:01:14.610218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:14.610181    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:14.669859 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:14.669796    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 17 16:01:15.903524 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:15.903495    2568 scope.go:117] "RemoveContainer" containerID="99c82f1f6b48ecc009c106041a445f8a0799e5639d6943928e807da99cfd1e3d"
Sep 17 16:01:19.610428 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:19.610379    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:20.591087 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:20.591044    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Sep 17 16:01:24.609741 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:24.609690    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:24.670475 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:24.670442    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 17 16:01:26.761451 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.761425    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:01:26.805744 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.805712    2568 generic.go:356] "Generic (PLEG): container finished" podID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerID="557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d" exitCode=0
Sep 17 16:01:26.805922 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.805771    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"
Sep 17 16:01:26.805922 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.805772    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" event={"ID":"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef","Type":"ContainerDied","Data":"557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d"}
Sep 17 16:01:26.805922 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.805899    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2" event={"ID":"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef","Type":"ContainerDied","Data":"02639f450f0104a61cca1be0b0b2c693c9a657d3063e52ed6ffb5a63ba54572e"}
Sep 17 16:01:26.806144 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.805939    2568 scope.go:117] "RemoveContainer" containerID="557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d"
Sep 17 16:01:26.814197 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.814180    2568 scope.go:117] "RemoveContainer" containerID="557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d"
Sep 17 16:01:26.814466 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:01:26.814443    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d\": container with ID starting with 557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d not found: ID does not exist" containerID="557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d"
Sep 17 16:01:26.814588 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.814471    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d"} err="failed to get container status \"557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d\": rpc error: code = NotFound desc = could not find container \"557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d\": container with ID starting with 557a70d41f9e64df9d4d19a4a5ea23c64b6d8fa9dbd64d418265d26537e7536d not found: ID does not exist"
Sep 17 16:01:26.927139 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.927071    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-openshift-service-ca-bundle\") pod \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") "
Sep 17 16:01:26.927273 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.927142    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls\") pod \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\" (UID: \"b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef\") "
Sep 17 16:01:26.927499 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.927470    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" (UID: "b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:01:26.929131 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:26.929110    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" (UID: "b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:01:27.027734 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:27.027707    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:27.027734 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:27.027731    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:27.129813 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:27.129782    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"]
Sep 17 16:01:27.135685 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:27.135656    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-8d4df-585ccf9548-ks5b2"]
Sep 17 16:01:27.830003 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:27.829968    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" path="/var/lib/kubelet/pods/b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef/volumes"
Sep 17 16:01:30.590997 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:30.590971    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:01:34.669908 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:34.669871    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Sep 17 16:01:42.902557 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902483    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"]
Sep 17 16:01:42.902937 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902919    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kube-rbac-proxy"
Sep 17 16:01:42.902985 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902938    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kube-rbac-proxy"
Sep 17 16:01:42.902985 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902947    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container"
Sep 17 16:01:42.902985 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902952    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container"
Sep 17 16:01:42.902985 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902963    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13"
Sep 17 16:01:42.902985 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902970    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13"
Sep 17 16:01:42.903218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902990    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df"
Sep 17 16:01:42.903218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.902995    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df"
Sep 17 16:01:42.903218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.903071    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="661f10dc-ec3d-4353-b855-f87f4a06b1f6" containerName="sequence-graph-9ed13"
Sep 17 16:01:42.903218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.903081    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kube-rbac-proxy"
Sep 17 16:01:42.903218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.903087    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d22eccc7-996a-4b17-9565-57d8016c2f6a" containerName="kserve-container"
Sep 17 16:01:42.903218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.903095    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c0d7a7-1181-4f27-90e2-d1fa0b6dc9ef" containerName="ensemble-graph-8d4df"
Sep 17 16:01:42.907616 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.907597    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:42.910172 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.910145    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-efa50-serving-cert\""
Sep 17 16:01:42.910285 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.910151    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-efa50-kube-rbac-proxy-sar-config\""
Sep 17 16:01:42.915870 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:42.915828    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"]
Sep 17 16:01:43.066177 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.066139    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cc8c35-2e96-45ce-afe4-d05b4598464f-openshift-service-ca-bundle\") pod \"sequence-graph-efa50-57f544b449-pwlbg\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") " pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.066328 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.066191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4cc8c35-2e96-45ce-afe4-d05b4598464f-proxy-tls\") pod \"sequence-graph-efa50-57f544b449-pwlbg\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") " pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.167305 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.167225    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cc8c35-2e96-45ce-afe4-d05b4598464f-openshift-service-ca-bundle\") pod \"sequence-graph-efa50-57f544b449-pwlbg\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") " pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.167305 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.167282    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4cc8c35-2e96-45ce-afe4-d05b4598464f-proxy-tls\") pod \"sequence-graph-efa50-57f544b449-pwlbg\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") " pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.167934 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.167901    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cc8c35-2e96-45ce-afe4-d05b4598464f-openshift-service-ca-bundle\") pod \"sequence-graph-efa50-57f544b449-pwlbg\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") " pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.169599 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.169577    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4cc8c35-2e96-45ce-afe4-d05b4598464f-proxy-tls\") pod \"sequence-graph-efa50-57f544b449-pwlbg\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") " pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.218357 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.218332    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.495733 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.495705    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"]
Sep 17 16:01:43.497388 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:01:43.497363    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cc8c35_2e96_45ce_afe4_d05b4598464f.slice/crio-feeb8e959d0749806855b00de07cde7a9f3f704b4b2342e99e7a6e1ea1c0c79d WatchSource:0}: Error finding container feeb8e959d0749806855b00de07cde7a9f3f704b4b2342e99e7a6e1ea1c0c79d: Status 404 returned error can't find the container with id feeb8e959d0749806855b00de07cde7a9f3f704b4b2342e99e7a6e1ea1c0c79d
Sep 17 16:01:43.892405 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.892318    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" event={"ID":"c4cc8c35-2e96-45ce-afe4-d05b4598464f","Type":"ContainerStarted","Data":"261d6f4458d008e0d1f9b5b0b65c3ec8730135bbef738c72cac6c1b6a0636c09"}
Sep 17 16:01:43.892562 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.892403    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" event={"ID":"c4cc8c35-2e96-45ce-afe4-d05b4598464f","Type":"ContainerStarted","Data":"feeb8e959d0749806855b00de07cde7a9f3f704b4b2342e99e7a6e1ea1c0c79d"}
Sep 17 16:01:43.892562 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.892448    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:43.914910 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:43.914864    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podStartSLOduration=1.9148305190000001 podStartE2EDuration="1.914830519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:01:43.913479794 +0000 UTC m=+1348.501524381" watchObservedRunningTime="2026-09-17 16:01:43.914830519 +0000 UTC m=+1348.502875105"
Sep 17 16:01:44.670987 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:44.670957    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:01:49.901890 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:49.901855    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:01:53.010221 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.010190    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"]
Sep 17 16:01:53.010728 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.010398    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50" containerID="cri-o://261d6f4458d008e0d1f9b5b0b65c3ec8730135bbef738c72cac6c1b6a0636c09" gracePeriod=30
Sep 17 16:01:53.109250 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.109208    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"]
Sep 17 16:01:53.110435 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.109533    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container" containerID="cri-o://8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba" gracePeriod=30
Sep 17 16:01:53.110435 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.109655    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kube-rbac-proxy" containerID="cri-o://434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7" gracePeriod=30
Sep 17 16:01:53.231505 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.231476    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"]
Sep 17 16:01:53.235037 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.235017    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.237583 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.237561    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-923c8-predictor-serving-cert\""
Sep 17 16:01:53.237834 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.237812    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-923c8-kube-rbac-proxy-sar-config\""
Sep 17 16:01:53.245509 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.245489    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"]
Sep 17 16:01:53.249860 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.249811    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.249969 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.249940    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctgnz\" (UniqueName: \"kubernetes.io/projected/3c709c30-b696-41a2-ac51-124a2c83066e-kube-api-access-ctgnz\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.250038 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.250002    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c709c30-b696-41a2-ac51-124a2c83066e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.350689 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.350594    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c709c30-b696-41a2-ac51-124a2c83066e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.350885 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.350678    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.350885 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.350734    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctgnz\" (UniqueName: \"kubernetes.io/projected/3c709c30-b696-41a2-ac51-124a2c83066e-kube-api-access-ctgnz\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.350885 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:01:53.350832    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-923c8-predictor-serving-cert: secret "success-200-isvc-923c8-predictor-serving-cert" not found
Sep 17 16:01:53.351084 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:01:53.350914    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls podName:3c709c30-b696-41a2-ac51-124a2c83066e nodeName:}" failed. No retries permitted until 2026-09-17 16:01:53.85089513 +0000 UTC m=+1358.438939702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls") pod "success-200-isvc-923c8-predictor-85cccd84fd-twbwf" (UID: "3c709c30-b696-41a2-ac51-124a2c83066e") : secret "success-200-isvc-923c8-predictor-serving-cert" not found
Sep 17 16:01:53.351388 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.351364    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c709c30-b696-41a2-ac51-124a2c83066e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.362685 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.362662    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctgnz\" (UniqueName: \"kubernetes.io/projected/3c709c30-b696-41a2-ac51-124a2c83066e-kube-api-access-ctgnz\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.854446 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.854408    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.856757 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.856734    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls\") pod \"success-200-isvc-923c8-predictor-85cccd84fd-twbwf\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") " pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:53.939069 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.939039    2568 generic.go:356] "Generic (PLEG): container finished" podID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerID="434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7" exitCode=2
Sep 17 16:01:53.939222 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:53.939090    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" event={"ID":"d7d5e861-fd6b-4e7e-b632-4531528ebf56","Type":"ContainerDied","Data":"434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7"}
Sep 17 16:01:54.146518 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.146435    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:54.420516 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.420494    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"]
Sep 17 16:01:54.422804 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:01:54.422774    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c709c30_b696_41a2_ac51_124a2c83066e.slice/crio-8fa26659da24e0979df727793369ec28d0c2e528cbdf8625364726283433634a WatchSource:0}: Error finding container 8fa26659da24e0979df727793369ec28d0c2e528cbdf8625364726283433634a: Status 404 returned error can't find the container with id 8fa26659da24e0979df727793369ec28d0c2e528cbdf8625364726283433634a
Sep 17 16:01:54.900263 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.900227    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:01:54.945547 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.945517    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" event={"ID":"3c709c30-b696-41a2-ac51-124a2c83066e","Type":"ContainerStarted","Data":"6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618"}
Sep 17 16:01:54.945716 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.945571    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" event={"ID":"3c709c30-b696-41a2-ac51-124a2c83066e","Type":"ContainerStarted","Data":"a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857"}
Sep 17 16:01:54.945716 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.945598    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" event={"ID":"3c709c30-b696-41a2-ac51-124a2c83066e","Type":"ContainerStarted","Data":"8fa26659da24e0979df727793369ec28d0c2e528cbdf8625364726283433634a"}
Sep 17 16:01:54.945716 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.945676    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:54.981764 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:54.981720    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podStartSLOduration=1.9817068820000001 podStartE2EDuration="1.981706882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:01:54.981035612 +0000 UTC m=+1359.569080198" watchObservedRunningTime="2026-09-17 16:01:54.981706882 +0000 UTC m=+1359.569751469"
Sep 17 16:01:55.585857 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:55.585808    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.39:8643/healthz\": dial tcp 10.134.0.39:8643: connect: connection refused"
Sep 17 16:01:55.950705 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:55.950675    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:01:55.952181 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:55.952152    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Sep 17 16:01:56.260523 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.260498    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:01:56.274974 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.274950    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbd98\" (UniqueName: \"kubernetes.io/projected/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-api-access-tbd98\") pod \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") "
Sep 17 16:01:56.275119 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.275020    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d7d5e861-fd6b-4e7e-b632-4531528ebf56-proxy-tls\") pod \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") "
Sep 17 16:01:56.275119 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.275042    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-rbac-proxy-sar-config\") pod \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\" (UID: \"d7d5e861-fd6b-4e7e-b632-4531528ebf56\") "
Sep 17 16:01:56.275427 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.275401    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d7d5e861-fd6b-4e7e-b632-4531528ebf56" (UID: "d7d5e861-fd6b-4e7e-b632-4531528ebf56"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:01:56.276988 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.276966    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-api-access-tbd98" (OuterVolumeSpecName: "kube-api-access-tbd98") pod "d7d5e861-fd6b-4e7e-b632-4531528ebf56" (UID: "d7d5e861-fd6b-4e7e-b632-4531528ebf56"). InnerVolumeSpecName "kube-api-access-tbd98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:01:56.277123 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.277105    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d5e861-fd6b-4e7e-b632-4531528ebf56-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d7d5e861-fd6b-4e7e-b632-4531528ebf56" (UID: "d7d5e861-fd6b-4e7e-b632-4531528ebf56"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:01:56.376374 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.376336    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:56.376374 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.376368    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbd98\" (UniqueName: \"kubernetes.io/projected/d7d5e861-fd6b-4e7e-b632-4531528ebf56-kube-api-access-tbd98\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:56.376374 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.376380    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d7d5e861-fd6b-4e7e-b632-4531528ebf56-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:01:56.905944 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.905912    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"]
Sep 17 16:01:56.906367 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.906331    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kube-rbac-proxy"
Sep 17 16:01:56.906367 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.906344    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kube-rbac-proxy"
Sep 17 16:01:56.906367 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.906360    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container"
Sep 17 16:01:56.906367 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.906366    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container"
Sep 17 16:01:56.906502 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.906439    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kserve-container"
Sep 17 16:01:56.906502 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.906450    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerName="kube-rbac-proxy"
Sep 17 16:01:56.909295 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.909278    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:56.911872 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.911832    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-bfbea-kube-rbac-proxy-sar-config\""
Sep 17 16:01:56.911872 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.911832    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-bfbea-serving-cert\""
Sep 17 16:01:56.918830 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.918810    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"]
Sep 17 16:01:56.956716 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.956682    2568 generic.go:356] "Generic (PLEG): container finished" podID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" containerID="8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba" exitCode=0
Sep 17 16:01:56.956896 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.956732    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"
Sep 17 16:01:56.956896 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.956767    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" event={"ID":"d7d5e861-fd6b-4e7e-b632-4531528ebf56","Type":"ContainerDied","Data":"8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba"}
Sep 17 16:01:56.956896 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.956869    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv" event={"ID":"d7d5e861-fd6b-4e7e-b632-4531528ebf56","Type":"ContainerDied","Data":"03be0f99cc175b7c5c1df99db24b0af37804d59f1e1d2aa03cb223396297dec6"}
Sep 17 16:01:56.957052 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.956902    2568 scope.go:117] "RemoveContainer" containerID="434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7"
Sep 17 16:01:56.957398 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.957370    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Sep 17 16:01:56.965425 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.965409    2568 scope.go:117] "RemoveContainer" containerID="8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba"
Sep 17 16:01:56.973952 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.973934    2568 scope.go:117] "RemoveContainer" containerID="434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7"
Sep 17 16:01:56.975522 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:01:56.975473    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7\": container with ID starting with 434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7 not found: ID does not exist" containerID="434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7"
Sep 17 16:01:56.975522 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.975510    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7"} err="failed to get container status \"434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7\": rpc error: code = NotFound desc = could not find container \"434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7\": container with ID starting with 434ceb8d3cb7d1be49fc6306067cfff6a116b0d5eb1ab4bf223469f1be2febb7 not found: ID does not exist"
Sep 17 16:01:56.975681 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.975536    2568 scope.go:117] "RemoveContainer" containerID="8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba"
Sep 17 16:01:56.976054 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:01:56.976033    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba\": container with ID starting with 8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba not found: ID does not exist" containerID="8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba"
Sep 17 16:01:56.976131 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.976059    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba"} err="failed to get container status \"8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba\": rpc error: code = NotFound desc = could not find container \"8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba\": container with ID starting with 8c9fd301e671cea975d77932bdb5f5db13658705546a5299c10c1359d6319dba not found: ID does not exist"
Sep 17 16:01:56.981236 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.981217    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"]
Sep 17 16:01:56.981496 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.981475    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae88b85-720d-4e55-ab64-e8fe71948f0d-openshift-service-ca-bundle\") pod \"ensemble-graph-bfbea-6dbb48967f-9lrwx\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") " pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:56.981549 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.981514    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae88b85-720d-4e55-ab64-e8fe71948f0d-proxy-tls\") pod \"ensemble-graph-bfbea-6dbb48967f-9lrwx\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") " pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:56.987103 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:56.987077    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-efa50-predictor-5f56d967f4-2f9vv"]
Sep 17 16:01:57.082741 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.082713    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae88b85-720d-4e55-ab64-e8fe71948f0d-openshift-service-ca-bundle\") pod \"ensemble-graph-bfbea-6dbb48967f-9lrwx\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") " pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:57.082914 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.082756    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae88b85-720d-4e55-ab64-e8fe71948f0d-proxy-tls\") pod \"ensemble-graph-bfbea-6dbb48967f-9lrwx\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") " pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:57.083336 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.083308    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae88b85-720d-4e55-ab64-e8fe71948f0d-openshift-service-ca-bundle\") pod \"ensemble-graph-bfbea-6dbb48967f-9lrwx\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") " pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:57.085090 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.085072    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae88b85-720d-4e55-ab64-e8fe71948f0d-proxy-tls\") pod \"ensemble-graph-bfbea-6dbb48967f-9lrwx\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") " pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:57.220552 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.220526    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:57.498402 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.498356    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"]
Sep 17 16:01:57.500510 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:01:57.500484    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae88b85_720d_4e55_ab64_e8fe71948f0d.slice/crio-b0f0b99e8391c1d9d4d1d1aed3c3acaba46bbae92c661a998ca87717f12c27ba WatchSource:0}: Error finding container b0f0b99e8391c1d9d4d1d1aed3c3acaba46bbae92c661a998ca87717f12c27ba: Status 404 returned error can't find the container with id b0f0b99e8391c1d9d4d1d1aed3c3acaba46bbae92c661a998ca87717f12c27ba
Sep 17 16:01:57.831628 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.831548    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d5e861-fd6b-4e7e-b632-4531528ebf56" path="/var/lib/kubelet/pods/d7d5e861-fd6b-4e7e-b632-4531528ebf56/volumes"
Sep 17 16:01:57.963752 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.963712    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" event={"ID":"5ae88b85-720d-4e55-ab64-e8fe71948f0d","Type":"ContainerStarted","Data":"c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748"}
Sep 17 16:01:57.964149 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.963774    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" event={"ID":"5ae88b85-720d-4e55-ab64-e8fe71948f0d","Type":"ContainerStarted","Data":"b0f0b99e8391c1d9d4d1d1aed3c3acaba46bbae92c661a998ca87717f12c27ba"}
Sep 17 16:01:57.964149 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.963803    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:01:57.984642 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:57.984599    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podStartSLOduration=1.984584634 podStartE2EDuration="1.984584634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:01:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:01:57.983315137 +0000 UTC m=+1362.571359724" watchObservedRunningTime="2026-09-17 16:01:57.984584634 +0000 UTC m=+1362.572629221"
Sep 17 16:01:59.900197 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:01:59.900160    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:02:01.962303 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:01.962271    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:02:01.962999 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:01.962969    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Sep 17 16:02:03.973712 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:03.973679    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:02:04.900806 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:04.900769    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:02:04.900995 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:04.900890    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:02:09.900792 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:09.900749    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:02:11.963065 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:11.963019    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Sep 17 16:02:14.901515 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:14.901468    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:02:19.900730 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:19.900694    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:02:21.963395 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:21.963357    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Sep 17 16:02:23.081528 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.081499    2568 generic.go:356] "Generic (PLEG): container finished" podID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerID="261d6f4458d008e0d1f9b5b0b65c3ec8730135bbef738c72cac6c1b6a0636c09" exitCode=0
Sep 17 16:02:23.081875 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.081583    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" event={"ID":"c4cc8c35-2e96-45ce-afe4-d05b4598464f","Type":"ContainerDied","Data":"261d6f4458d008e0d1f9b5b0b65c3ec8730135bbef738c72cac6c1b6a0636c09"}
Sep 17 16:02:23.164367 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.164344    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:02:23.319197 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.319096    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cc8c35-2e96-45ce-afe4-d05b4598464f-openshift-service-ca-bundle\") pod \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") "
Sep 17 16:02:23.319373 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.319196    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4cc8c35-2e96-45ce-afe4-d05b4598464f-proxy-tls\") pod \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\" (UID: \"c4cc8c35-2e96-45ce-afe4-d05b4598464f\") "
Sep 17 16:02:23.319492 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.319469    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4cc8c35-2e96-45ce-afe4-d05b4598464f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c4cc8c35-2e96-45ce-afe4-d05b4598464f" (UID: "c4cc8c35-2e96-45ce-afe4-d05b4598464f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:02:23.321390 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.321361    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4cc8c35-2e96-45ce-afe4-d05b4598464f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c4cc8c35-2e96-45ce-afe4-d05b4598464f" (UID: "c4cc8c35-2e96-45ce-afe4-d05b4598464f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:02:23.419896 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.419859    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4cc8c35-2e96-45ce-afe4-d05b4598464f-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:02:23.419896 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:23.419889    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cc8c35-2e96-45ce-afe4-d05b4598464f-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:02:24.087182 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:24.087148    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"
Sep 17 16:02:24.087682 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:24.087146    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg" event={"ID":"c4cc8c35-2e96-45ce-afe4-d05b4598464f","Type":"ContainerDied","Data":"feeb8e959d0749806855b00de07cde7a9f3f704b4b2342e99e7a6e1ea1c0c79d"}
Sep 17 16:02:24.087682 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:24.087295    2568 scope.go:117] "RemoveContainer" containerID="261d6f4458d008e0d1f9b5b0b65c3ec8730135bbef738c72cac6c1b6a0636c09"
Sep 17 16:02:24.106910 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:24.106882    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"]
Sep 17 16:02:24.111632 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:24.111609    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-efa50-57f544b449-pwlbg"]
Sep 17 16:02:25.831689 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:25.831650    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" path="/var/lib/kubelet/pods/c4cc8c35-2e96-45ce-afe4-d05b4598464f/volumes"
Sep 17 16:02:31.963339 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:31.963299    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.43:8080: connect: connection refused"
Sep 17 16:02:41.963646 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:41.963618    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:02:53.290128 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.290090    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"]
Sep 17 16:02:53.290542 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.290447    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50"
Sep 17 16:02:53.290542 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.290458    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50"
Sep 17 16:02:53.290625 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.290559    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4cc8c35-2e96-45ce-afe4-d05b4598464f" containerName="sequence-graph-efa50"
Sep 17 16:02:53.293710 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.293692    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.296484 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.296462    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-923c8-serving-cert\""
Sep 17 16:02:53.296769 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.296753    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-923c8-kube-rbac-proxy-sar-config\""
Sep 17 16:02:53.312373 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.312350    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"]
Sep 17 16:02:53.373680 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.373644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9ef005b-e653-4e7e-aef0-652391790c57-proxy-tls\") pod \"sequence-graph-923c8-f9f5fbfb6-8zcjx\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") " pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.373862 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.373695    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ef005b-e653-4e7e-aef0-652391790c57-openshift-service-ca-bundle\") pod \"sequence-graph-923c8-f9f5fbfb6-8zcjx\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") " pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.475218 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.475175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9ef005b-e653-4e7e-aef0-652391790c57-proxy-tls\") pod \"sequence-graph-923c8-f9f5fbfb6-8zcjx\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") " pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.475393 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.475231    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ef005b-e653-4e7e-aef0-652391790c57-openshift-service-ca-bundle\") pod \"sequence-graph-923c8-f9f5fbfb6-8zcjx\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") " pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.475818 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.475791    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ef005b-e653-4e7e-aef0-652391790c57-openshift-service-ca-bundle\") pod \"sequence-graph-923c8-f9f5fbfb6-8zcjx\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") " pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.477594 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.477575    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9ef005b-e653-4e7e-aef0-652391790c57-proxy-tls\") pod \"sequence-graph-923c8-f9f5fbfb6-8zcjx\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") " pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.604805 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.604696    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:53.880313 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:53.880288    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"]
Sep 17 16:02:53.882640 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:02:53.882612    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ef005b_e653_4e7e_aef0_652391790c57.slice/crio-29c78b6d311132175619014cff9c960609cf819dd29860aa636b69b482610df1 WatchSource:0}: Error finding container 29c78b6d311132175619014cff9c960609cf819dd29860aa636b69b482610df1: Status 404 returned error can't find the container with id 29c78b6d311132175619014cff9c960609cf819dd29860aa636b69b482610df1
Sep 17 16:02:54.229949 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:54.229914    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" event={"ID":"e9ef005b-e653-4e7e-aef0-652391790c57","Type":"ContainerStarted","Data":"8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6"}
Sep 17 16:02:54.230139 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:54.229974    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" event={"ID":"e9ef005b-e653-4e7e-aef0-652391790c57","Type":"ContainerStarted","Data":"29c78b6d311132175619014cff9c960609cf819dd29860aa636b69b482610df1"}
Sep 17 16:02:54.230139 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:54.230001    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:02:54.251771 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:02:54.251710    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podStartSLOduration=1.251690665 podStartE2EDuration="1.251690665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:02:54.251426887 +0000 UTC m=+1418.839471477" watchObservedRunningTime="2026-09-17 16:02:54.251690665 +0000 UTC m=+1418.839735253"
Sep 17 16:03:00.239626 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:03:00.239597    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:04:15.878630 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:04:15.878600    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:04:15.882168 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:04:15.882146    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:09:15.908513 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:09:15.908476    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:09:15.912169 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:09:15.912136    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:10:11.568375 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.568337    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"]
Sep 17 16:10:11.568966 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.568565    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea" containerID="cri-o://c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748" gracePeriod=30
Sep 17 16:10:11.763852 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.763811    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"]
Sep 17 16:10:11.764174 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.764117    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" containerID="cri-o://a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5" gracePeriod=30
Sep 17 16:10:11.764313 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.764182    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kube-rbac-proxy" containerID="cri-o://6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd" gracePeriod=30
Sep 17 16:10:11.869770 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.869695    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"]
Sep 17 16:10:11.874160 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.874132    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:11.877624 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.877598    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5a441-kube-rbac-proxy-sar-config\""
Sep 17 16:10:11.877955 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.877932    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5a441-predictor-serving-cert\""
Sep 17 16:10:11.888011 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.887988    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"]
Sep 17 16:10:11.946102 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.946064    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04103e42-744d-4aba-ab30-08b5a01984d2-proxy-tls\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:11.946316 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.946133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04103e42-744d-4aba-ab30-08b5a01984d2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:11.946316 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:11.946191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rnqm\" (UniqueName: \"kubernetes.io/projected/04103e42-744d-4aba-ab30-08b5a01984d2-kube-api-access-2rnqm\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.047339 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.047310    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04103e42-744d-4aba-ab30-08b5a01984d2-proxy-tls\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.047493 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.047355    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04103e42-744d-4aba-ab30-08b5a01984d2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.047493 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.047393    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rnqm\" (UniqueName: \"kubernetes.io/projected/04103e42-744d-4aba-ab30-08b5a01984d2-kube-api-access-2rnqm\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.048110 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.048090    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04103e42-744d-4aba-ab30-08b5a01984d2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.049757 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.049739    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04103e42-744d-4aba-ab30-08b5a01984d2-proxy-tls\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.065411 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.065387    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rnqm\" (UniqueName: \"kubernetes.io/projected/04103e42-744d-4aba-ab30-08b5a01984d2-kube-api-access-2rnqm\") pod \"success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") " pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.185613 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.185533    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:12.225905 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.225739    2568 generic.go:356] "Generic (PLEG): container finished" podID="198a5a38-4c86-43c8-8327-55470532dafd" containerID="6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd" exitCode=2
Sep 17 16:10:12.225905 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.225815    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" event={"ID":"198a5a38-4c86-43c8-8327-55470532dafd","Type":"ContainerDied","Data":"6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd"}
Sep 17 16:10:12.470571 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:12.470543    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"]
Sep 17 16:10:12.472805 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:10:12.472778    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04103e42_744d_4aba_ab30_08b5a01984d2.slice/crio-e4db4bc681072cec04027b59dcda0ff237fffdafba4d7f3e65bf34fe57c040a5 WatchSource:0}: Error finding container e4db4bc681072cec04027b59dcda0ff237fffdafba4d7f3e65bf34fe57c040a5: Status 404 returned error can't find the container with id e4db4bc681072cec04027b59dcda0ff237fffdafba4d7f3e65bf34fe57c040a5
Sep 17 16:10:13.232452 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:13.232412    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" event={"ID":"04103e42-744d-4aba-ab30-08b5a01984d2","Type":"ContainerStarted","Data":"73e9d368308ada8138dea689fc59d0101c2970729ad3f714f20d70648c6e1822"}
Sep 17 16:10:13.232994 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:13.232472    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" event={"ID":"04103e42-744d-4aba-ab30-08b5a01984d2","Type":"ContainerStarted","Data":"cdac5a7fb6b1c552c7b4437b577cb5d0d5e98fc785914f854ac81953fb8dd8c3"}
Sep 17 16:10:13.232994 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:13.232492    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" event={"ID":"04103e42-744d-4aba-ab30-08b5a01984d2","Type":"ContainerStarted","Data":"e4db4bc681072cec04027b59dcda0ff237fffdafba4d7f3e65bf34fe57c040a5"}
Sep 17 16:10:13.232994 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:13.232553    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:13.259383 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:13.259324    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podStartSLOduration=2.259308441 podStartE2EDuration="2.259308441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:10:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:10:13.257924619 +0000 UTC m=+1857.845969207" watchObservedRunningTime="2026-09-17 16:10:13.259308441 +0000 UTC m=+1857.847353071"
Sep 17 16:10:13.972154 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:13.972117    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:10:14.237379 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.237287    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:14.238655 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.238623    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Sep 17 16:10:14.712100 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.712076    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:10:14.771364 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.771341    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a5a38-4c86-43c8-8327-55470532dafd-kube-rbac-proxy-sar-config\") pod \"198a5a38-4c86-43c8-8327-55470532dafd\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") "
Sep 17 16:10:14.771511 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.771500    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a5a38-4c86-43c8-8327-55470532dafd-proxy-tls\") pod \"198a5a38-4c86-43c8-8327-55470532dafd\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") "
Sep 17 16:10:14.771561 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.771530    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xrcm\" (UniqueName: \"kubernetes.io/projected/198a5a38-4c86-43c8-8327-55470532dafd-kube-api-access-7xrcm\") pod \"198a5a38-4c86-43c8-8327-55470532dafd\" (UID: \"198a5a38-4c86-43c8-8327-55470532dafd\") "
Sep 17 16:10:14.771664 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.771644    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198a5a38-4c86-43c8-8327-55470532dafd-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "198a5a38-4c86-43c8-8327-55470532dafd" (UID: "198a5a38-4c86-43c8-8327-55470532dafd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:10:14.771773 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.771754    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a5a38-4c86-43c8-8327-55470532dafd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:10:14.773480 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.773458    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/198a5a38-4c86-43c8-8327-55470532dafd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "198a5a38-4c86-43c8-8327-55470532dafd" (UID: "198a5a38-4c86-43c8-8327-55470532dafd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:10:14.773612 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.773565    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198a5a38-4c86-43c8-8327-55470532dafd-kube-api-access-7xrcm" (OuterVolumeSpecName: "kube-api-access-7xrcm") pod "198a5a38-4c86-43c8-8327-55470532dafd" (UID: "198a5a38-4c86-43c8-8327-55470532dafd"). InnerVolumeSpecName "kube-api-access-7xrcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:10:14.873133 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.873067    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a5a38-4c86-43c8-8327-55470532dafd-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:10:14.873133 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:14.873091    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xrcm\" (UniqueName: \"kubernetes.io/projected/198a5a38-4c86-43c8-8327-55470532dafd-kube-api-access-7xrcm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:10:15.243206 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.243173    2568 generic.go:356] "Generic (PLEG): container finished" podID="198a5a38-4c86-43c8-8327-55470532dafd" containerID="a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5" exitCode=0
Sep 17 16:10:15.243641 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.243213    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"
Sep 17 16:10:15.243641 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.243247    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" event={"ID":"198a5a38-4c86-43c8-8327-55470532dafd","Type":"ContainerDied","Data":"a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5"}
Sep 17 16:10:15.243641 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.243305    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" event={"ID":"198a5a38-4c86-43c8-8327-55470532dafd","Type":"ContainerDied","Data":"d397660dfdd59f33d66b3a48f5c095c194150a38ccf03ba0ac949a34fc219e57"}
Sep 17 16:10:15.243641 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.243330    2568 scope.go:117] "RemoveContainer" containerID="6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd"
Sep 17 16:10:15.243857 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.243810    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Sep 17 16:10:15.252907 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.252888    2568 scope.go:117] "RemoveContainer" containerID="a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5"
Sep 17 16:10:15.260355 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.260340    2568 scope.go:117] "RemoveContainer" containerID="6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd"
Sep 17 16:10:15.260593 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:10:15.260573    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd\": container with ID starting with 6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd not found: ID does not exist" containerID="6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd"
Sep 17 16:10:15.260648 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.260600    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd"} err="failed to get container status \"6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd\": rpc error: code = NotFound desc = could not find container \"6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd\": container with ID starting with 6f589116d0934d8b059a13031b1bfbdc4d4f750fedd68d8b81523e583d0f3acd not found: ID does not exist"
Sep 17 16:10:15.260648 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.260617    2568 scope.go:117] "RemoveContainer" containerID="a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5"
Sep 17 16:10:15.260895 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:10:15.260822    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5\": container with ID starting with a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5 not found: ID does not exist" containerID="a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5"
Sep 17 16:10:15.260895 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.260862    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5"} err="failed to get container status \"a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5\": rpc error: code = NotFound desc = could not find container \"a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5\": container with ID starting with a4fd7310f48518419a82d6af2c2c66b76abaf437ef5b178d78f5a2d0df9711a5 not found: ID does not exist"
Sep 17 16:10:15.278552 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.278530    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"]
Sep 17 16:10:15.286992 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.286973    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c"]
Sep 17 16:10:15.665421 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.665323    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.41:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 17 16:10:15.671517 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.671477    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-bfbea-predictor-db56bc584-klx7c" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: i/o timeout"
Sep 17 16:10:15.838638 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:15.838608    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="198a5a38-4c86-43c8-8327-55470532dafd" path="/var/lib/kubelet/pods/198a5a38-4c86-43c8-8327-55470532dafd/volumes"
Sep 17 16:10:18.972287 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:18.972249    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:10:20.248913 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:20.248887    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:10:20.249315 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:20.249254    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Sep 17 16:10:23.971884 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:23.971834    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:10:23.972248 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:23.971941    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:10:28.972546 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:28.972502    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:10:30.249578 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:30.249529    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Sep 17 16:10:33.971624 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:33.971584    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:10:38.972793 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:38.972707    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:10:40.249443 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:40.249401    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Sep 17 16:10:41.720592 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:41.720562    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:10:41.907052 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:41.906970    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae88b85-720d-4e55-ab64-e8fe71948f0d-openshift-service-ca-bundle\") pod \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") "
Sep 17 16:10:41.907052 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:41.907005    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae88b85-720d-4e55-ab64-e8fe71948f0d-proxy-tls\") pod \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\" (UID: \"5ae88b85-720d-4e55-ab64-e8fe71948f0d\") "
Sep 17 16:10:41.909756 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:41.907981    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae88b85-720d-4e55-ab64-e8fe71948f0d-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5ae88b85-720d-4e55-ab64-e8fe71948f0d" (UID: "5ae88b85-720d-4e55-ab64-e8fe71948f0d"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:10:41.915518 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:41.915441    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae88b85-720d-4e55-ab64-e8fe71948f0d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5ae88b85-720d-4e55-ab64-e8fe71948f0d" (UID: "5ae88b85-720d-4e55-ab64-e8fe71948f0d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:10:42.009241 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.009213    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae88b85-720d-4e55-ab64-e8fe71948f0d-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:10:42.009241 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.009238    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae88b85-720d-4e55-ab64-e8fe71948f0d-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:10:42.369866 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.369809    2568 generic.go:356] "Generic (PLEG): container finished" podID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerID="c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748" exitCode=0
Sep 17 16:10:42.370075 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.369877    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"
Sep 17 16:10:42.370075 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.369880    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" event={"ID":"5ae88b85-720d-4e55-ab64-e8fe71948f0d","Type":"ContainerDied","Data":"c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748"}
Sep 17 16:10:42.370075 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.369967    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx" event={"ID":"5ae88b85-720d-4e55-ab64-e8fe71948f0d","Type":"ContainerDied","Data":"b0f0b99e8391c1d9d4d1d1aed3c3acaba46bbae92c661a998ca87717f12c27ba"}
Sep 17 16:10:42.370075 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.369991    2568 scope.go:117] "RemoveContainer" containerID="c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748"
Sep 17 16:10:42.378711 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.378695    2568 scope.go:117] "RemoveContainer" containerID="c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748"
Sep 17 16:10:42.379040 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:10:42.379017    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748\": container with ID starting with c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748 not found: ID does not exist" containerID="c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748"
Sep 17 16:10:42.379097 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.379049    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748"} err="failed to get container status \"c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748\": rpc error: code = NotFound desc = could not find container \"c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748\": container with ID starting with c3aae843fe88f7e20b09c18fd2d25d3a16240d474f9e6b8fa678f6592078d748 not found: ID does not exist"
Sep 17 16:10:42.393739 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.393711    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"]
Sep 17 16:10:42.397624 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:42.397592    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-bfbea-6dbb48967f-9lrwx"]
Sep 17 16:10:43.831097 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:43.831061    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" path="/var/lib/kubelet/pods/5ae88b85-720d-4e55-ab64-e8fe71948f0d/volumes"
Sep 17 16:10:50.249322 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:10:50.249283    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.46:8080: connect: connection refused"
Sep 17 16:11:00.250401 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:00.250373    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:11:07.847601 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:07.847562    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"]
Sep 17 16:11:07.848015 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:07.847736    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8" containerID="cri-o://8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6" gracePeriod=30
Sep 17 16:11:07.981966 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:07.981935    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"]
Sep 17 16:11:07.982153 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:07.982126    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container" containerID="cri-o://a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857" gracePeriod=30
Sep 17 16:11:07.982242 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:07.982184    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kube-rbac-proxy" containerID="cri-o://6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618" gracePeriod=30
Sep 17 16:11:08.151323 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151291    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"]
Sep 17 16:11:08.151690 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151677    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea"
Sep 17 16:11:08.151746 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151692    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea"
Sep 17 16:11:08.151746 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151701    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container"
Sep 17 16:11:08.151746 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151707    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container"
Sep 17 16:11:08.151746 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151716    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kube-rbac-proxy"
Sep 17 16:11:08.151746 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151722    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kube-rbac-proxy"
Sep 17 16:11:08.151922 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151781    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kserve-container"
Sep 17 16:11:08.151922 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151791    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="198a5a38-4c86-43c8-8327-55470532dafd" containerName="kube-rbac-proxy"
Sep 17 16:11:08.151922 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.151797    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae88b85-720d-4e55-ab64-e8fe71948f0d" containerName="ensemble-graph-bfbea"
Sep 17 16:11:08.154817 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.154796    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.157402 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.157375    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ece61-predictor-serving-cert\""
Sep 17 16:11:08.157862 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.157828    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ece61-kube-rbac-proxy-sar-config\""
Sep 17 16:11:08.182874 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.182831    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"]
Sep 17 16:11:08.223145 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.223112    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smghp\" (UniqueName: \"kubernetes.io/projected/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-api-access-smghp\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.223398 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.223171    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.223398 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.223284    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.324347 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.324306    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.324544 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.324390    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smghp\" (UniqueName: \"kubernetes.io/projected/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-api-access-smghp\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.324544 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.324429    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.324544 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:11:08.324468    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-ece61-predictor-serving-cert: secret "success-200-isvc-ece61-predictor-serving-cert" not found
Sep 17 16:11:08.324544 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:11:08.324532    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls podName:1897a425-3a0f-4ade-9909-cf203f4898ee nodeName:}" failed. No retries permitted until 2026-09-17 16:11:08.824514719 +0000 UTC m=+1913.412559283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls") pod "success-200-isvc-ece61-predictor-5bff789676-cz8x9" (UID: "1897a425-3a0f-4ade-9909-cf203f4898ee") : secret "success-200-isvc-ece61-predictor-serving-cert" not found
Sep 17 16:11:08.325135 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.325112    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.335271 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.335250    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smghp\" (UniqueName: \"kubernetes.io/projected/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-api-access-smghp\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.491358 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.491319    2568 generic.go:356] "Generic (PLEG): container finished" podID="3c709c30-b696-41a2-ac51-124a2c83066e" containerID="6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618" exitCode=2
Sep 17 16:11:08.491511 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.491384    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" event={"ID":"3c709c30-b696-41a2-ac51-124a2c83066e","Type":"ContainerDied","Data":"6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618"}
Sep 17 16:11:08.829889 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.829767    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:08.832292 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:08.832269    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls\") pod \"success-200-isvc-ece61-predictor-5bff789676-cz8x9\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") " pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:09.065929 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:09.065887    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:09.343358 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:09.343298    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"]
Sep 17 16:11:09.345891 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:11:09.345860    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1897a425_3a0f_4ade_9909_cf203f4898ee.slice/crio-d948c45fb5133ef181dea7d6172b8a785dd119f36d8d00e8f7782ca67820be16 WatchSource:0}: Error finding container d948c45fb5133ef181dea7d6172b8a785dd119f36d8d00e8f7782ca67820be16: Status 404 returned error can't find the container with id d948c45fb5133ef181dea7d6172b8a785dd119f36d8d00e8f7782ca67820be16
Sep 17 16:11:09.498145 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:09.498116    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" event={"ID":"1897a425-3a0f-4ade-9909-cf203f4898ee","Type":"ContainerStarted","Data":"9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b"}
Sep 17 16:11:09.498272 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:09.498175    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" event={"ID":"1897a425-3a0f-4ade-9909-cf203f4898ee","Type":"ContainerStarted","Data":"3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c"}
Sep 17 16:11:09.498272 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:09.498198    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" event={"ID":"1897a425-3a0f-4ade-9909-cf203f4898ee","Type":"ContainerStarted","Data":"d948c45fb5133ef181dea7d6172b8a785dd119f36d8d00e8f7782ca67820be16"}
Sep 17 16:11:09.498272 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:09.498227    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:09.526446 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:09.526399    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podStartSLOduration=1.526383337 podStartE2EDuration="1.526383337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:11:09.524763463 +0000 UTC m=+1914.112808051" watchObservedRunningTime="2026-09-17 16:11:09.526383337 +0000 UTC m=+1914.114427923"
Sep 17 16:11:10.238699 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:10.238659    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:10.502250 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:10.502164    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:10.503603 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:10.503570    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Sep 17 16:11:11.125664 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.125639    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:11:11.253480 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.253446    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls\") pod \"3c709c30-b696-41a2-ac51-124a2c83066e\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") "
Sep 17 16:11:11.253961 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.253487    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c709c30-b696-41a2-ac51-124a2c83066e-kube-rbac-proxy-sar-config\") pod \"3c709c30-b696-41a2-ac51-124a2c83066e\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") "
Sep 17 16:11:11.253961 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.253516    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctgnz\" (UniqueName: \"kubernetes.io/projected/3c709c30-b696-41a2-ac51-124a2c83066e-kube-api-access-ctgnz\") pod \"3c709c30-b696-41a2-ac51-124a2c83066e\" (UID: \"3c709c30-b696-41a2-ac51-124a2c83066e\") "
Sep 17 16:11:11.254073 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.254047    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c709c30-b696-41a2-ac51-124a2c83066e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3c709c30-b696-41a2-ac51-124a2c83066e" (UID: "3c709c30-b696-41a2-ac51-124a2c83066e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:11:11.255658 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.255628    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3c709c30-b696-41a2-ac51-124a2c83066e" (UID: "3c709c30-b696-41a2-ac51-124a2c83066e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:11:11.255658 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.255629    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c709c30-b696-41a2-ac51-124a2c83066e-kube-api-access-ctgnz" (OuterVolumeSpecName: "kube-api-access-ctgnz") pod "3c709c30-b696-41a2-ac51-124a2c83066e" (UID: "3c709c30-b696-41a2-ac51-124a2c83066e"). InnerVolumeSpecName "kube-api-access-ctgnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:11:11.354955 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.354925    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c709c30-b696-41a2-ac51-124a2c83066e-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:11.354955 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.354952    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c709c30-b696-41a2-ac51-124a2c83066e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:11.355144 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.354967    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctgnz\" (UniqueName: \"kubernetes.io/projected/3c709c30-b696-41a2-ac51-124a2c83066e-kube-api-access-ctgnz\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:11.508484 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.508392    2568 generic.go:356] "Generic (PLEG): container finished" podID="3c709c30-b696-41a2-ac51-124a2c83066e" containerID="a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857" exitCode=0
Sep 17 16:11:11.508484 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.508443    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"
Sep 17 16:11:11.508484 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.508472    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" event={"ID":"3c709c30-b696-41a2-ac51-124a2c83066e","Type":"ContainerDied","Data":"a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857"}
Sep 17 16:11:11.508760 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.508525    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf" event={"ID":"3c709c30-b696-41a2-ac51-124a2c83066e","Type":"ContainerDied","Data":"8fa26659da24e0979df727793369ec28d0c2e528cbdf8625364726283433634a"}
Sep 17 16:11:11.508760 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.508552    2568 scope.go:117] "RemoveContainer" containerID="6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618"
Sep 17 16:11:11.509171 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.509139    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Sep 17 16:11:11.517339 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.517315    2568 scope.go:117] "RemoveContainer" containerID="a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857"
Sep 17 16:11:11.524667 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.524650    2568 scope.go:117] "RemoveContainer" containerID="6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618"
Sep 17 16:11:11.524929 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:11:11.524911    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618\": container with ID starting with 6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618 not found: ID does not exist" containerID="6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618"
Sep 17 16:11:11.524970 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.524939    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618"} err="failed to get container status \"6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618\": rpc error: code = NotFound desc = could not find container \"6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618\": container with ID starting with 6b7943973b649af6d6a1d7ab041b39f7614f412cfa2cc35c948081585d252618 not found: ID does not exist"
Sep 17 16:11:11.524970 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.524956    2568 scope.go:117] "RemoveContainer" containerID="a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857"
Sep 17 16:11:11.525171 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:11:11.525157    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857\": container with ID starting with a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857 not found: ID does not exist" containerID="a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857"
Sep 17 16:11:11.525209 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.525176    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857"} err="failed to get container status \"a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857\": rpc error: code = NotFound desc = could not find container \"a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857\": container with ID starting with a956fe5120134a9d211052ed8e3b9eda2251b41a288eb9a27e0206f667bee857 not found: ID does not exist"
Sep 17 16:11:11.533918 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.533895    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"]
Sep 17 16:11:11.541081 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.541061    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-923c8-predictor-85cccd84fd-twbwf"]
Sep 17 16:11:11.813786 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.813713    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"]
Sep 17 16:11:11.814109 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.814096    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container"
Sep 17 16:11:11.814157 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.814111    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container"
Sep 17 16:11:11.814157 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.814124    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kube-rbac-proxy"
Sep 17 16:11:11.814157 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.814131    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kube-rbac-proxy"
Sep 17 16:11:11.814253 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.814186    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kube-rbac-proxy"
Sep 17 16:11:11.814253 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.814199    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" containerName="kserve-container"
Sep 17 16:11:11.818593 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.818567    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:11.820940 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.820911    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-5a441-kube-rbac-proxy-sar-config\""
Sep 17 16:11:11.821091 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.820994    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-5a441-serving-cert\""
Sep 17 16:11:11.833727 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.833691    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c709c30-b696-41a2-ac51-124a2c83066e" path="/var/lib/kubelet/pods/3c709c30-b696-41a2-ac51-124a2c83066e/volumes"
Sep 17 16:11:11.834157 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.834135    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"]
Sep 17 16:11:11.959414 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.959384    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-proxy-tls\") pod \"splitter-graph-5a441-666945ddb8-dswfr\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") " pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:11.959592 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:11.959426    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-openshift-service-ca-bundle\") pod \"splitter-graph-5a441-666945ddb8-dswfr\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") " pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:12.060144 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.060106    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-proxy-tls\") pod \"splitter-graph-5a441-666945ddb8-dswfr\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") " pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:12.060144 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.060147    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-openshift-service-ca-bundle\") pod \"splitter-graph-5a441-666945ddb8-dswfr\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") " pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:12.060861 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.060774    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-openshift-service-ca-bundle\") pod \"splitter-graph-5a441-666945ddb8-dswfr\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") " pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:12.062601 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.062577    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-proxy-tls\") pod \"splitter-graph-5a441-666945ddb8-dswfr\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") " pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:12.130698 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.130637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:12.407947 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.407904    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"]
Sep 17 16:11:12.410120 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:11:12.410091    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac5e7b3_d7e2_4a35_8d64_bdda870f1859.slice/crio-405697a4c14c16d5491c0f197a4feddcef7780d601ab70c865fdd7d1a6bfb3f8 WatchSource:0}: Error finding container 405697a4c14c16d5491c0f197a4feddcef7780d601ab70c865fdd7d1a6bfb3f8: Status 404 returned error can't find the container with id 405697a4c14c16d5491c0f197a4feddcef7780d601ab70c865fdd7d1a6bfb3f8
Sep 17 16:11:12.514901 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.514857    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" event={"ID":"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859","Type":"ContainerStarted","Data":"a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8"}
Sep 17 16:11:12.515062 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.514919    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" event={"ID":"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859","Type":"ContainerStarted","Data":"405697a4c14c16d5491c0f197a4feddcef7780d601ab70c865fdd7d1a6bfb3f8"}
Sep 17 16:11:12.515062 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.514942    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:12.553580 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:12.553524    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podStartSLOduration=1.553507865 podStartE2EDuration="1.553507865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:11:12.552977568 +0000 UTC m=+1917.141022155" watchObservedRunningTime="2026-09-17 16:11:12.553507865 +0000 UTC m=+1917.141552453"
Sep 17 16:11:15.238655 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:15.238617    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:16.513927 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:16.513895    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:11:16.514298 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:16.514250    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Sep 17 16:11:18.531587 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:18.531555    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:20.238910 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:20.238872    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:20.239313 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:20.238982    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:11:21.967439 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:21.967405    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"]
Sep 17 16:11:21.967898 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:21.967590    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441" containerID="cri-o://a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8" gracePeriod=30
Sep 17 16:11:22.217062 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.217027    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"]
Sep 17 16:11:22.217278 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.217242    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container" containerID="cri-o://cdac5a7fb6b1c552c7b4437b577cb5d0d5e98fc785914f854ac81953fb8dd8c3" gracePeriod=30
Sep 17 16:11:22.217367 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.217292    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kube-rbac-proxy" containerID="cri-o://73e9d368308ada8138dea689fc59d0101c2970729ad3f714f20d70648c6e1822" gracePeriod=30
Sep 17 16:11:22.345399 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.345366    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"]
Sep 17 16:11:22.349071 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.349051    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.352774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.352753    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8fed6-kube-rbac-proxy-sar-config\""
Sep 17 16:11:22.352874 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.352753    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8fed6-predictor-serving-cert\""
Sep 17 16:11:22.377780 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.377754    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"]
Sep 17 16:11:22.449850 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.449781    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrlft\" (UniqueName: \"kubernetes.io/projected/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-api-access-hrlft\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.449850 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.449828    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-proxy-tls\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.450094 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.449983    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.551289 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.551206    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.551289 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.551268    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrlft\" (UniqueName: \"kubernetes.io/projected/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-api-access-hrlft\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.551531 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.551293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-proxy-tls\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.551895 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.551862    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.553666 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.553649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-proxy-tls\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.563952 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.563922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrlft\" (UniqueName: \"kubernetes.io/projected/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-api-access-hrlft\") pod \"success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") " pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.568889 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.568866    2568 generic.go:356] "Generic (PLEG): container finished" podID="04103e42-744d-4aba-ab30-08b5a01984d2" containerID="73e9d368308ada8138dea689fc59d0101c2970729ad3f714f20d70648c6e1822" exitCode=2
Sep 17 16:11:22.569004 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.568913    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" event={"ID":"04103e42-744d-4aba-ab30-08b5a01984d2","Type":"ContainerDied","Data":"73e9d368308ada8138dea689fc59d0101c2970729ad3f714f20d70648c6e1822"}
Sep 17 16:11:22.659517 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.659488    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:22.939107 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:22.939076    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"]
Sep 17 16:11:22.939950 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:11:22.939915    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3603d6_a1f5_4fd8_bde9_b4cad6f9ccb6.slice/crio-9421218dc4be1d312d0e524c8df0c1f45496900566bff7be3a5f05dd226df571 WatchSource:0}: Error finding container 9421218dc4be1d312d0e524c8df0c1f45496900566bff7be3a5f05dd226df571: Status 404 returned error can't find the container with id 9421218dc4be1d312d0e524c8df0c1f45496900566bff7be3a5f05dd226df571
Sep 17 16:11:23.530540 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:23.530499    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:23.575629 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:23.575596    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" event={"ID":"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6","Type":"ContainerStarted","Data":"31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd"}
Sep 17 16:11:23.575790 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:23.575676    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" event={"ID":"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6","Type":"ContainerStarted","Data":"82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583"}
Sep 17 16:11:23.575790 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:23.575698    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" event={"ID":"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6","Type":"ContainerStarted","Data":"9421218dc4be1d312d0e524c8df0c1f45496900566bff7be3a5f05dd226df571"}
Sep 17 16:11:23.575790 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:23.575721    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:24.580324 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:24.580292    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:24.581882 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:24.581832    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.49:8080: connect: connection refused"
Sep 17 16:11:25.240153 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.240115    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:25.244471 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.244445    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.46:8643/healthz\": dial tcp 10.134.0.46:8643: connect: connection refused"
Sep 17 16:11:25.587951 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.587920    2568 generic.go:356] "Generic (PLEG): container finished" podID="04103e42-744d-4aba-ab30-08b5a01984d2" containerID="cdac5a7fb6b1c552c7b4437b577cb5d0d5e98fc785914f854ac81953fb8dd8c3" exitCode=0
Sep 17 16:11:25.588369 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.588004    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" event={"ID":"04103e42-744d-4aba-ab30-08b5a01984d2","Type":"ContainerDied","Data":"cdac5a7fb6b1c552c7b4437b577cb5d0d5e98fc785914f854ac81953fb8dd8c3"}
Sep 17 16:11:25.588369 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.588278    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.49:8080: connect: connection refused"
Sep 17 16:11:25.668785 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.668759    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:11:25.707092 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.707045    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podStartSLOduration=3.707029216 podStartE2EDuration="3.707029216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:11:23.61834159 +0000 UTC m=+1928.206386177" watchObservedRunningTime="2026-09-17 16:11:25.707029216 +0000 UTC m=+1930.295073803"
Sep 17 16:11:25.781869 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.781820    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04103e42-744d-4aba-ab30-08b5a01984d2-kube-rbac-proxy-sar-config\") pod \"04103e42-744d-4aba-ab30-08b5a01984d2\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") "
Sep 17 16:11:25.782015 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.781883    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04103e42-744d-4aba-ab30-08b5a01984d2-proxy-tls\") pod \"04103e42-744d-4aba-ab30-08b5a01984d2\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") "
Sep 17 16:11:25.782015 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.781928    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rnqm\" (UniqueName: \"kubernetes.io/projected/04103e42-744d-4aba-ab30-08b5a01984d2-kube-api-access-2rnqm\") pod \"04103e42-744d-4aba-ab30-08b5a01984d2\" (UID: \"04103e42-744d-4aba-ab30-08b5a01984d2\") "
Sep 17 16:11:25.782256 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.782216    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04103e42-744d-4aba-ab30-08b5a01984d2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "04103e42-744d-4aba-ab30-08b5a01984d2" (UID: "04103e42-744d-4aba-ab30-08b5a01984d2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:11:25.783921 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.783901    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04103e42-744d-4aba-ab30-08b5a01984d2-kube-api-access-2rnqm" (OuterVolumeSpecName: "kube-api-access-2rnqm") pod "04103e42-744d-4aba-ab30-08b5a01984d2" (UID: "04103e42-744d-4aba-ab30-08b5a01984d2"). InnerVolumeSpecName "kube-api-access-2rnqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:11:25.784038 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.783979    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04103e42-744d-4aba-ab30-08b5a01984d2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "04103e42-744d-4aba-ab30-08b5a01984d2" (UID: "04103e42-744d-4aba-ab30-08b5a01984d2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:11:25.883244 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.883214    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/04103e42-744d-4aba-ab30-08b5a01984d2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:25.883244 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.883240    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04103e42-744d-4aba-ab30-08b5a01984d2-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:25.883404 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:25.883255    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rnqm\" (UniqueName: \"kubernetes.io/projected/04103e42-744d-4aba-ab30-08b5a01984d2-kube-api-access-2rnqm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:26.514500 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:26.514461    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Sep 17 16:11:26.594038 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:26.594008    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"
Sep 17 16:11:26.594038 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:26.594025    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww" event={"ID":"04103e42-744d-4aba-ab30-08b5a01984d2","Type":"ContainerDied","Data":"e4db4bc681072cec04027b59dcda0ff237fffdafba4d7f3e65bf34fe57c040a5"}
Sep 17 16:11:26.594462 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:26.594086    2568 scope.go:117] "RemoveContainer" containerID="73e9d368308ada8138dea689fc59d0101c2970729ad3f714f20d70648c6e1822"
Sep 17 16:11:26.602313 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:26.602293    2568 scope.go:117] "RemoveContainer" containerID="cdac5a7fb6b1c552c7b4437b577cb5d0d5e98fc785914f854ac81953fb8dd8c3"
Sep 17 16:11:26.635221 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:26.635192    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"]
Sep 17 16:11:26.637874 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:26.637851    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5a441-predictor-6c9d84f75d-8c9ww"]
Sep 17 16:11:27.831760 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:27.831718    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" path="/var/lib/kubelet/pods/04103e42-744d-4aba-ab30-08b5a01984d2/volumes"
Sep 17 16:11:28.530690 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:28.530647    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:30.238827 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:30.238778    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:30.593409 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:30.593314    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:11:30.593990 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:30.593958    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.49:8080: connect: connection refused"
Sep 17 16:11:33.530987 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:33.530942    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:33.531484 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:33.531077    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:35.239558 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:35.239501    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:36.514606 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:36.514564    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Sep 17 16:11:37.992472 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:37.992447    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:11:38.088691 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.088661    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ef005b-e653-4e7e-aef0-652391790c57-openshift-service-ca-bundle\") pod \"e9ef005b-e653-4e7e-aef0-652391790c57\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") "
Sep 17 16:11:38.088832 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.088717    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9ef005b-e653-4e7e-aef0-652391790c57-proxy-tls\") pod \"e9ef005b-e653-4e7e-aef0-652391790c57\" (UID: \"e9ef005b-e653-4e7e-aef0-652391790c57\") "
Sep 17 16:11:38.089022 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.089001    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9ef005b-e653-4e7e-aef0-652391790c57-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e9ef005b-e653-4e7e-aef0-652391790c57" (UID: "e9ef005b-e653-4e7e-aef0-652391790c57"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:11:38.090766 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.090745    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9ef005b-e653-4e7e-aef0-652391790c57-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e9ef005b-e653-4e7e-aef0-652391790c57" (UID: "e9ef005b-e653-4e7e-aef0-652391790c57"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:11:38.189560 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.189531    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ef005b-e653-4e7e-aef0-652391790c57-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:38.189560 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.189559    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9ef005b-e653-4e7e-aef0-652391790c57-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:38.531174 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.531068    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:38.652868 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.652818    2568 generic.go:356] "Generic (PLEG): container finished" podID="e9ef005b-e653-4e7e-aef0-652391790c57" containerID="8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6" exitCode=0
Sep 17 16:11:38.653039 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.652873    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" event={"ID":"e9ef005b-e653-4e7e-aef0-652391790c57","Type":"ContainerDied","Data":"8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6"}
Sep 17 16:11:38.653039 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.652926    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx" event={"ID":"e9ef005b-e653-4e7e-aef0-652391790c57","Type":"ContainerDied","Data":"29c78b6d311132175619014cff9c960609cf819dd29860aa636b69b482610df1"}
Sep 17 16:11:38.653039 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.652954    2568 scope.go:117] "RemoveContainer" containerID="8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6"
Sep 17 16:11:38.653039 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.652883    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"
Sep 17 16:11:38.662112 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.662094    2568 scope.go:117] "RemoveContainer" containerID="8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6"
Sep 17 16:11:38.662385 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:11:38.662365    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6\": container with ID starting with 8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6 not found: ID does not exist" containerID="8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6"
Sep 17 16:11:38.662438 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.662393    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6"} err="failed to get container status \"8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6\": rpc error: code = NotFound desc = could not find container \"8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6\": container with ID starting with 8b48ca923be99909e51ab31da5c92c8ec20b961e1e43b45fd215927672e3b7f6 not found: ID does not exist"
Sep 17 16:11:38.692828 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.692802    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"]
Sep 17 16:11:38.696950 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:38.696931    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-923c8-f9f5fbfb6-8zcjx"]
Sep 17 16:11:39.833421 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:39.833387    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" path="/var/lib/kubelet/pods/e9ef005b-e653-4e7e-aef0-652391790c57/volumes"
Sep 17 16:11:40.594093 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:40.594055    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.49:8080: connect: connection refused"
Sep 17 16:11:43.537386 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:43.537340    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:46.514975 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:46.514933    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.47:8080: connect: connection refused"
Sep 17 16:11:48.531023 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:48.530984    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:11:50.594497 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:50.594460    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.49:8080: connect: connection refused"
Sep 17 16:11:52.622926 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.622898    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:52.711664 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.711627    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-proxy-tls\") pod \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") "
Sep 17 16:11:52.711859 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.711737    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-openshift-service-ca-bundle\") pod \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\" (UID: \"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859\") "
Sep 17 16:11:52.712118 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.712088    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" (UID: "3ac5e7b3-d7e2-4a35-8d64-bdda870f1859"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:11:52.713722 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.713699    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" (UID: "3ac5e7b3-d7e2-4a35-8d64-bdda870f1859"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:11:52.721198 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.721175    2568 generic.go:356] "Generic (PLEG): container finished" podID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerID="a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8" exitCode=0
Sep 17 16:11:52.721326 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.721221    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"
Sep 17 16:11:52.721326 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.721254    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" event={"ID":"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859","Type":"ContainerDied","Data":"a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8"}
Sep 17 16:11:52.721326 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.721309    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr" event={"ID":"3ac5e7b3-d7e2-4a35-8d64-bdda870f1859","Type":"ContainerDied","Data":"405697a4c14c16d5491c0f197a4feddcef7780d601ab70c865fdd7d1a6bfb3f8"}
Sep 17 16:11:52.721441 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.721332    2568 scope.go:117] "RemoveContainer" containerID="a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8"
Sep 17 16:11:52.730091 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.730069    2568 scope.go:117] "RemoveContainer" containerID="a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8"
Sep 17 16:11:52.730436 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:11:52.730407    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8\": container with ID starting with a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8 not found: ID does not exist" containerID="a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8"
Sep 17 16:11:52.730538 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.730447    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8"} err="failed to get container status \"a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8\": rpc error: code = NotFound desc = could not find container \"a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8\": container with ID starting with a8d1ca99449abe9aacf5540e26de90b50e329931217797316523f235b0651dc8 not found: ID does not exist"
Sep 17 16:11:52.749310 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.749235    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"]
Sep 17 16:11:52.761076 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.761047    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-5a441-666945ddb8-dswfr"]
Sep 17 16:11:52.812619 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.812584    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:52.812619 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:52.812613    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:11:53.831551 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:53.831517    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" path="/var/lib/kubelet/pods/3ac5e7b3-d7e2-4a35-8d64-bdda870f1859/volumes"
Sep 17 16:11:56.514874 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:11:56.514822    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:12:00.594906 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:00.594867    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.49:8080: connect: connection refused"
Sep 17 16:12:08.139890 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.139848    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"]
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140231    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kube-rbac-proxy"
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140243    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kube-rbac-proxy"
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140258    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container"
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140265    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container"
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140278    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8"
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140284    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8"
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140290    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441"
Sep 17 16:12:08.140300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140295    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441"
Sep 17 16:12:08.140552 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140344    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9ef005b-e653-4e7e-aef0-652391790c57" containerName="sequence-graph-923c8"
Sep 17 16:12:08.140552 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140353    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kube-rbac-proxy"
Sep 17 16:12:08.140552 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140360    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ac5e7b3-d7e2-4a35-8d64-bdda870f1859" containerName="splitter-graph-5a441"
Sep 17 16:12:08.140552 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.140367    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="04103e42-744d-4aba-ab30-08b5a01984d2" containerName="kserve-container"
Sep 17 16:12:08.144626 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.144604    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.148791 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.148765    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-ece61-kube-rbac-proxy-sar-config\""
Sep 17 16:12:08.148905 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.148765    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-ece61-serving-cert\""
Sep 17 16:12:08.154463 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.154441    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"]
Sep 17 16:12:08.245171 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.245136    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-openshift-service-ca-bundle\") pod \"switch-graph-ece61-7ffdf9795f-9g9x7\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") " pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.245349 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.245202    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-proxy-tls\") pod \"switch-graph-ece61-7ffdf9795f-9g9x7\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") " pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.346408 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.346369    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-openshift-service-ca-bundle\") pod \"switch-graph-ece61-7ffdf9795f-9g9x7\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") " pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.346581 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.346428    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-proxy-tls\") pod \"switch-graph-ece61-7ffdf9795f-9g9x7\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") " pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.347084 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.347048    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-openshift-service-ca-bundle\") pod \"switch-graph-ece61-7ffdf9795f-9g9x7\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") " pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.348787 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.348763    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-proxy-tls\") pod \"switch-graph-ece61-7ffdf9795f-9g9x7\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") " pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.456165 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.456133    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:08.734189 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.734158    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"]
Sep 17 16:12:08.736594 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:12:08.736566    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ffd2d2_e9ab_46cd_9614_6763638c57fb.slice/crio-333edfad2549aa7de6afe4dfa538db9289388b595e90ab8d8beac8b869e7e039 WatchSource:0}: Error finding container 333edfad2549aa7de6afe4dfa538db9289388b595e90ab8d8beac8b869e7e039: Status 404 returned error can't find the container with id 333edfad2549aa7de6afe4dfa538db9289388b595e90ab8d8beac8b869e7e039
Sep 17 16:12:08.804745 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:08.804718    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" event={"ID":"c2ffd2d2-e9ab-46cd-9614-6763638c57fb","Type":"ContainerStarted","Data":"333edfad2549aa7de6afe4dfa538db9289388b595e90ab8d8beac8b869e7e039"}
Sep 17 16:12:09.811434 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:09.811400    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" event={"ID":"c2ffd2d2-e9ab-46cd-9614-6763638c57fb","Type":"ContainerStarted","Data":"2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c"}
Sep 17 16:12:09.811809 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:09.811461    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:09.839460 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:09.839414    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podStartSLOduration=1.8393983280000001 podStartE2EDuration="1.839398328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:12:09.838662996 +0000 UTC m=+1974.426707585" watchObservedRunningTime="2026-09-17 16:12:09.839398328 +0000 UTC m=+1974.427442916"
Sep 17 16:12:10.595037 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:10.595001    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:12:15.821551 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:15.821523    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:12:22.326638 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.326600    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"]
Sep 17 16:12:22.329899 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.329878    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:22.332324 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.332303    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-8fed6-kube-rbac-proxy-sar-config\""
Sep 17 16:12:22.332708 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.332687    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-8fed6-serving-cert\""
Sep 17 16:12:22.346251 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.346227    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"]
Sep 17 16:12:22.474073 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.474037    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a5d139-4398-4a2a-9879-ffaab383dda7-openshift-service-ca-bundle\") pod \"splitter-graph-8fed6-6cdf7f47c9-xn6d2\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") " pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:22.474241 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.474098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls\") pod \"splitter-graph-8fed6-6cdf7f47c9-xn6d2\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") " pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:22.575383 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.575346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a5d139-4398-4a2a-9879-ffaab383dda7-openshift-service-ca-bundle\") pod \"splitter-graph-8fed6-6cdf7f47c9-xn6d2\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") " pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:22.575556 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.575416    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls\") pod \"splitter-graph-8fed6-6cdf7f47c9-xn6d2\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") " pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:22.575556 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:12:22.575532    2568 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-8fed6-serving-cert: secret "splitter-graph-8fed6-serving-cert" not found
Sep 17 16:12:22.575629 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:12:22.575597    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls podName:35a5d139-4398-4a2a-9879-ffaab383dda7 nodeName:}" failed. No retries permitted until 2026-09-17 16:12:23.075581381 +0000 UTC m=+1987.663625946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls") pod "splitter-graph-8fed6-6cdf7f47c9-xn6d2" (UID: "35a5d139-4398-4a2a-9879-ffaab383dda7") : secret "splitter-graph-8fed6-serving-cert" not found
Sep 17 16:12:22.575985 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:22.575964    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a5d139-4398-4a2a-9879-ffaab383dda7-openshift-service-ca-bundle\") pod \"splitter-graph-8fed6-6cdf7f47c9-xn6d2\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") " pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:23.080566 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.080523    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls\") pod \"splitter-graph-8fed6-6cdf7f47c9-xn6d2\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") " pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:23.083028 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.082998    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls\") pod \"splitter-graph-8fed6-6cdf7f47c9-xn6d2\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") " pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:23.239944 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.239910    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:23.520590 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.520561    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"]
Sep 17 16:12:23.522606 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:12:23.522575    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a5d139_4398_4a2a_9879_ffaab383dda7.slice/crio-d7d4f9b76606713f8f903e7f378b775a78be9d817c0f0c9468d1b9900e77d8fd WatchSource:0}: Error finding container d7d4f9b76606713f8f903e7f378b775a78be9d817c0f0c9468d1b9900e77d8fd: Status 404 returned error can't find the container with id d7d4f9b76606713f8f903e7f378b775a78be9d817c0f0c9468d1b9900e77d8fd
Sep 17 16:12:23.879383 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.879299    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" event={"ID":"35a5d139-4398-4a2a-9879-ffaab383dda7","Type":"ContainerStarted","Data":"ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc"}
Sep 17 16:12:23.879383 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.879348    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" event={"ID":"35a5d139-4398-4a2a-9879-ffaab383dda7","Type":"ContainerStarted","Data":"d7d4f9b76606713f8f903e7f378b775a78be9d817c0f0c9468d1b9900e77d8fd"}
Sep 17 16:12:23.879556 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.879418    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:12:23.906781 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:23.906722    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podStartSLOduration=1.9067059469999998 podStartE2EDuration="1.906705947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:12:23.905126046 +0000 UTC m=+1988.493170633" watchObservedRunningTime="2026-09-17 16:12:23.906705947 +0000 UTC m=+1988.494750540"
Sep 17 16:12:29.889468 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:12:29.889438    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:14:15.937996 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:14:15.937966    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:14:15.943455 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:14:15.943430    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:19:15.967791 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:19:15.967760    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:19:15.973826 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:19:15.973805    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:20:36.967691 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:36.967655    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"]
Sep 17 16:20:36.968230 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:36.967884    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6" containerID="cri-o://ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc" gracePeriod=30
Sep 17 16:20:37.135045 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:37.135007    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"]
Sep 17 16:20:37.135300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:37.135251    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container" containerID="cri-o://82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583" gracePeriod=30
Sep 17 16:20:37.135377 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:37.135268    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kube-rbac-proxy" containerID="cri-o://31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd" gracePeriod=30
Sep 17 16:20:38.152603 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:38.152570    2568 generic.go:356] "Generic (PLEG): container finished" podID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerID="31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd" exitCode=2
Sep 17 16:20:38.153020 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:38.152641    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" event={"ID":"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6","Type":"ContainerDied","Data":"31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd"}
Sep 17 16:20:39.888382 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:39.888345    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:20:39.974542 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:39.974517    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:20:40.096439 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.096345    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-proxy-tls\") pod \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") "
Sep 17 16:20:40.096439 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.096432    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-rbac-proxy-sar-config\") pod \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") "
Sep 17 16:20:40.096645 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.096488    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrlft\" (UniqueName: \"kubernetes.io/projected/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-api-access-hrlft\") pod \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\" (UID: \"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6\") "
Sep 17 16:20:40.096810 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.096789    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" (UID: "8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:20:40.098445 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.098410    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" (UID: "8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:20:40.098543 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.098513    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-api-access-hrlft" (OuterVolumeSpecName: "kube-api-access-hrlft") pod "8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" (UID: "8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6"). InnerVolumeSpecName "kube-api-access-hrlft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:20:40.164270 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.164233    2568 generic.go:356] "Generic (PLEG): container finished" podID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerID="82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583" exitCode=0
Sep 17 16:20:40.164412 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.164287    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"
Sep 17 16:20:40.164412 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.164321    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" event={"ID":"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6","Type":"ContainerDied","Data":"82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583"}
Sep 17 16:20:40.164412 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.164384    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb" event={"ID":"8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6","Type":"ContainerDied","Data":"9421218dc4be1d312d0e524c8df0c1f45496900566bff7be3a5f05dd226df571"}
Sep 17 16:20:40.164412 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.164411    2568 scope.go:117] "RemoveContainer" containerID="31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd"
Sep 17 16:20:40.173421 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.173404    2568 scope.go:117] "RemoveContainer" containerID="82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583"
Sep 17 16:20:40.180773 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.180758    2568 scope.go:117] "RemoveContainer" containerID="31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd"
Sep 17 16:20:40.181055 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:20:40.181034    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd\": container with ID starting with 31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd not found: ID does not exist" containerID="31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd"
Sep 17 16:20:40.181110 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.181064    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd"} err="failed to get container status \"31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd\": rpc error: code = NotFound desc = could not find container \"31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd\": container with ID starting with 31cd8a453d7be114a426db6e02191ffdb45c9ac240594405a09fe6aad949d5dd not found: ID does not exist"
Sep 17 16:20:40.181110 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.181081    2568 scope.go:117] "RemoveContainer" containerID="82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583"
Sep 17 16:20:40.181331 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:20:40.181313    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583\": container with ID starting with 82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583 not found: ID does not exist" containerID="82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583"
Sep 17 16:20:40.181376 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.181337    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583"} err="failed to get container status \"82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583\": rpc error: code = NotFound desc = could not find container \"82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583\": container with ID starting with 82c288f5c80f1c7c3b971098e0fcdfd44b55325801c7955b42f80e13e9772583 not found: ID does not exist"
Sep 17 16:20:40.189898 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.189865    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"]
Sep 17 16:20:40.193980 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.193960    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8fed6-predictor-8697ddcc86-ldvnb"]
Sep 17 16:20:40.197535 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.197511    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrlft\" (UniqueName: \"kubernetes.io/projected/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-api-access-hrlft\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:20:40.197535 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.197532    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:20:40.197672 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:40.197544    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:20:41.833812 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:41.833776    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" path="/var/lib/kubelet/pods/8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6/volumes"
Sep 17 16:20:44.887816 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:44.887772    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:20:49.887909 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:49.887871    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:20:49.888286 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:49.887973    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:20:54.888113 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:54.888069    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:20:59.887856 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:20:59.887796    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:21:04.888309 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:04.888272    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:21:07.116860 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.116823    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:21:07.224930 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.224829    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls\") pod \"35a5d139-4398-4a2a-9879-ffaab383dda7\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") "
Sep 17 16:21:07.225092 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.224968    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a5d139-4398-4a2a-9879-ffaab383dda7-openshift-service-ca-bundle\") pod \"35a5d139-4398-4a2a-9879-ffaab383dda7\" (UID: \"35a5d139-4398-4a2a-9879-ffaab383dda7\") "
Sep 17 16:21:07.225318 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.225294    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35a5d139-4398-4a2a-9879-ffaab383dda7-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "35a5d139-4398-4a2a-9879-ffaab383dda7" (UID: "35a5d139-4398-4a2a-9879-ffaab383dda7"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:21:07.226668 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.226647    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "35a5d139-4398-4a2a-9879-ffaab383dda7" (UID: "35a5d139-4398-4a2a-9879-ffaab383dda7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:21:07.292668 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.292638    2568 generic.go:356] "Generic (PLEG): container finished" podID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerID="ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc" exitCode=0
Sep 17 16:21:07.292795 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.292672    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" event={"ID":"35a5d139-4398-4a2a-9879-ffaab383dda7","Type":"ContainerDied","Data":"ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc"}
Sep 17 16:21:07.292795 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.292686    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"
Sep 17 16:21:07.292795 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.292705    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2" event={"ID":"35a5d139-4398-4a2a-9879-ffaab383dda7","Type":"ContainerDied","Data":"d7d4f9b76606713f8f903e7f378b775a78be9d817c0f0c9468d1b9900e77d8fd"}
Sep 17 16:21:07.292795 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.292726    2568 scope.go:117] "RemoveContainer" containerID="ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc"
Sep 17 16:21:07.302826 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.302710    2568 scope.go:117] "RemoveContainer" containerID="ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc"
Sep 17 16:21:07.303122 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:21:07.303095    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc\": container with ID starting with ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc not found: ID does not exist" containerID="ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc"
Sep 17 16:21:07.303197 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.303130    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc"} err="failed to get container status \"ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc\": rpc error: code = NotFound desc = could not find container \"ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc\": container with ID starting with ca2ddd86e21d7bf4d1e3434131ea10ea62d292c82ff086b95168c7aa451dabfc not found: ID does not exist"
Sep 17 16:21:07.322201 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.322180    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"]
Sep 17 16:21:07.325189 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.325169    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-8fed6-6cdf7f47c9-xn6d2"]
Sep 17 16:21:07.325478 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.325465    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35a5d139-4398-4a2a-9879-ffaab383dda7-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:21:07.325531 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.325480    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a5d139-4398-4a2a-9879-ffaab383dda7-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:21:07.831065 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:21:07.831033    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" path="/var/lib/kubelet/pods/35a5d139-4398-4a2a-9879-ffaab383dda7/volumes"
Sep 17 16:24:15.997385 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:24:15.997285    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:24:16.007051 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:24:16.007030    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:28:27.473179 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:27.473140    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"]
Sep 17 16:28:27.473738 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:27.473350    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61" containerID="cri-o://2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c" gracePeriod=30
Sep 17 16:28:27.608818 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:27.608784    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"]
Sep 17 16:28:27.609101 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:27.609067    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container" containerID="cri-o://3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c" gracePeriod=30
Sep 17 16:28:27.609235 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:27.609104    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kube-rbac-proxy" containerID="cri-o://9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b" gracePeriod=30
Sep 17 16:28:28.136615 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:28.136529    2568 generic.go:356] "Generic (PLEG): container finished" podID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerID="9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b" exitCode=2
Sep 17 16:28:28.136615 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:28.136599    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" event={"ID":"1897a425-3a0f-4ade-9909-cf203f4898ee","Type":"ContainerDied","Data":"9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b"}
Sep 17 16:28:30.760822 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.760798    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:28:30.820289 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.820200    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:28:30.904195 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.904156    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smghp\" (UniqueName: \"kubernetes.io/projected/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-api-access-smghp\") pod \"1897a425-3a0f-4ade-9909-cf203f4898ee\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") "
Sep 17 16:28:30.904366 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.904235    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls\") pod \"1897a425-3a0f-4ade-9909-cf203f4898ee\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") "
Sep 17 16:28:30.904366 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.904272    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-rbac-proxy-sar-config\") pod \"1897a425-3a0f-4ade-9909-cf203f4898ee\" (UID: \"1897a425-3a0f-4ade-9909-cf203f4898ee\") "
Sep 17 16:28:30.904740 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.904710    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1897a425-3a0f-4ade-9909-cf203f4898ee" (UID: "1897a425-3a0f-4ade-9909-cf203f4898ee"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:28:30.906257 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.906233    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-api-access-smghp" (OuterVolumeSpecName: "kube-api-access-smghp") pod "1897a425-3a0f-4ade-9909-cf203f4898ee" (UID: "1897a425-3a0f-4ade-9909-cf203f4898ee"). InnerVolumeSpecName "kube-api-access-smghp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:28:30.906359 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:30.906330    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1897a425-3a0f-4ade-9909-cf203f4898ee" (UID: "1897a425-3a0f-4ade-9909-cf203f4898ee"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:28:31.005676 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.005639    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1897a425-3a0f-4ade-9909-cf203f4898ee-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:28:31.005676 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.005668    2568 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:28:31.005676 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.005678    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smghp\" (UniqueName: \"kubernetes.io/projected/1897a425-3a0f-4ade-9909-cf203f4898ee-kube-api-access-smghp\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:28:31.151742 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.151649    2568 generic.go:356] "Generic (PLEG): container finished" podID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerID="3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c" exitCode=0
Sep 17 16:28:31.151742 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.151701    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"
Sep 17 16:28:31.151742 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.151724    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" event={"ID":"1897a425-3a0f-4ade-9909-cf203f4898ee","Type":"ContainerDied","Data":"3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c"}
Sep 17 16:28:31.152034 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.151779    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9" event={"ID":"1897a425-3a0f-4ade-9909-cf203f4898ee","Type":"ContainerDied","Data":"d948c45fb5133ef181dea7d6172b8a785dd119f36d8d00e8f7782ca67820be16"}
Sep 17 16:28:31.152034 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.151804    2568 scope.go:117] "RemoveContainer" containerID="9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b"
Sep 17 16:28:31.161184 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.161165    2568 scope.go:117] "RemoveContainer" containerID="3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c"
Sep 17 16:28:31.168541 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.168524    2568 scope.go:117] "RemoveContainer" containerID="9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b"
Sep 17 16:28:31.168792 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:28:31.168775    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b\": container with ID starting with 9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b not found: ID does not exist" containerID="9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b"
Sep 17 16:28:31.168862 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.168801    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b"} err="failed to get container status \"9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b\": rpc error: code = NotFound desc = could not find container \"9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b\": container with ID starting with 9626bad51e74a4806bcc40d93b24100fad5fe4325e091ed5ef423ed134eef72b not found: ID does not exist"
Sep 17 16:28:31.168862 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.168818    2568 scope.go:117] "RemoveContainer" containerID="3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c"
Sep 17 16:28:31.169077 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:28:31.169057    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c\": container with ID starting with 3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c not found: ID does not exist" containerID="3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c"
Sep 17 16:28:31.169118 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.169083    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c"} err="failed to get container status \"3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c\": rpc error: code = NotFound desc = could not find container \"3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c\": container with ID starting with 3bad8657e6df39d9d65365d097dc01a8be7630e1cf9dc2611291d53ee50e963c not found: ID does not exist"
Sep 17 16:28:31.177117 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.177097    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"]
Sep 17 16:28:31.181447 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.181421    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ece61-predictor-5bff789676-cz8x9"]
Sep 17 16:28:31.831277 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:31.831244    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" path="/var/lib/kubelet/pods/1897a425-3a0f-4ade-9909-cf203f4898ee/volumes"
Sep 17 16:28:35.820385 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:35.820347    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:28:40.820195 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:40.820103    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:28:40.820562 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:40.820222    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:28:45.283052 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:45.283017    2568 ???:1] "http: TLS handshake error from 10.0.140.183:49230: EOF"
Sep 17 16:28:45.290688 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:45.290664    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:45.819929 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:45.819889    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:28:46.254857 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:46.254803    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:47.210230 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:47.210187    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:48.143038 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:48.142999    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:49.046079 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:49.046048    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:49.984667 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:49.984625    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:50.820272 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:50.820234    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:28:50.935635 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:50.935604    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:51.886963 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:51.886935    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:52.766140 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:52.766112    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:53.689300 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:53.689268    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:54.595676 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:54.595646    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:55.526635 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:55.526603    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:55.820366 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:55.820279    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:28:56.525399 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:56.525363    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:57.454046 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:57.454017    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ece61-7ffdf9795f-9g9x7_c2ffd2d2-e9ab-46cd-9614-6763638c57fb/switch-graph-ece61/0.log"
Sep 17 16:28:58.123723 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.123698    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:28:58.236980 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.236946    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-proxy-tls\") pod \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") "
Sep 17 16:28:58.237228 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.237027    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-openshift-service-ca-bundle\") pod \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\" (UID: \"c2ffd2d2-e9ab-46cd-9614-6763638c57fb\") "
Sep 17 16:28:58.237387 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.237362    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c2ffd2d2-e9ab-46cd-9614-6763638c57fb" (UID: "c2ffd2d2-e9ab-46cd-9614-6763638c57fb"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:28:58.239119 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.239062    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c2ffd2d2-e9ab-46cd-9614-6763638c57fb" (UID: "c2ffd2d2-e9ab-46cd-9614-6763638c57fb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:28:58.269768 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.269736    2568 generic.go:356] "Generic (PLEG): container finished" podID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerID="2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c" exitCode=0
Sep 17 16:28:58.269901 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.269781    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"
Sep 17 16:28:58.269901 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.269825    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" event={"ID":"c2ffd2d2-e9ab-46cd-9614-6763638c57fb","Type":"ContainerDied","Data":"2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c"}
Sep 17 16:28:58.269901 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.269898    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7" event={"ID":"c2ffd2d2-e9ab-46cd-9614-6763638c57fb","Type":"ContainerDied","Data":"333edfad2549aa7de6afe4dfa538db9289388b595e90ab8d8beac8b869e7e039"}
Sep 17 16:28:58.270031 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.269921    2568 scope.go:117] "RemoveContainer" containerID="2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c"
Sep 17 16:28:58.278615 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.278596    2568 scope.go:117] "RemoveContainer" containerID="2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c"
Sep 17 16:28:58.278870 ip-10-0-140-183 kubenswrapper[2568]: E0917 16:28:58.278831    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c\": container with ID starting with 2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c not found: ID does not exist" containerID="2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c"
Sep 17 16:28:58.278969 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.278875    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c"} err="failed to get container status \"2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c\": rpc error: code = NotFound desc = could not find container \"2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c\": container with ID starting with 2efe2ca2b7d32c8bcf7d8c734e5b25d555adf8865b790276e3ba826a5043e33c not found: ID does not exist"
Sep 17 16:28:58.295035 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.295006    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"]
Sep 17 16:28:58.299269 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.299248    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ece61-7ffdf9795f-9g9x7"]
Sep 17 16:28:58.338201 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.338179    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-openshift-service-ca-bundle\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:28:58.338201 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:58.338199    2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2ffd2d2-e9ab-46cd-9614-6763638c57fb-proxy-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Sep 17 16:28:59.830688 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:28:59.830655    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" path="/var/lib/kubelet/pods/c2ffd2d2-e9ab-46cd-9614-6763638c57fb/volumes"
Sep 17 16:29:00.936051 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936017    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-72p88/must-gather-mqnbt"]
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936332    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kube-rbac-proxy"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936353    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kube-rbac-proxy"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936368    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936374    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936382    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936389    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936397    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kube-rbac-proxy"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936403    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kube-rbac-proxy"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936416    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936421    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container"
Sep 17 16:29:00.936424 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936430    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container"
Sep 17 16:29:00.936774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936436    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container"
Sep 17 16:29:00.936774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936488    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2ffd2d2-e9ab-46cd-9614-6763638c57fb" containerName="switch-graph-ece61"
Sep 17 16:29:00.936774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936495    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kube-rbac-proxy"
Sep 17 16:29:00.936774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936503    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a5d139-4398-4a2a-9879-ffaab383dda7" containerName="splitter-graph-8fed6"
Sep 17 16:29:00.936774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936509    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kserve-container"
Sep 17 16:29:00.936774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936514    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c3603d6-a1f5-4fd8-bde9-b4cad6f9ccb6" containerName="kube-rbac-proxy"
Sep 17 16:29:00.936774 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.936524    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1897a425-3a0f-4ade-9909-cf203f4898ee" containerName="kserve-container"
Sep 17 16:29:00.941123 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.941104    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:00.952882 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.952849    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-72p88\"/\"default-dockercfg-vh8vv\""
Sep 17 16:29:00.953003 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.952884    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-72p88\"/\"openshift-service-ca.crt\""
Sep 17 16:29:00.953239 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.953218    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-72p88\"/\"kube-root-ca.crt\""
Sep 17 16:29:00.974377 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:00.974353    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72p88/must-gather-mqnbt"]
Sep 17 16:29:01.064147 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.064098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e75db8d7-1181-4cca-830d-569476a2a890-must-gather-output\") pod \"must-gather-mqnbt\" (UID: \"e75db8d7-1181-4cca-830d-569476a2a890\") " pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:01.064333 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.064160    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqmp\" (UniqueName: \"kubernetes.io/projected/e75db8d7-1181-4cca-830d-569476a2a890-kube-api-access-nzqmp\") pod \"must-gather-mqnbt\" (UID: \"e75db8d7-1181-4cca-830d-569476a2a890\") " pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:01.164541 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.164503    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e75db8d7-1181-4cca-830d-569476a2a890-must-gather-output\") pod \"must-gather-mqnbt\" (UID: \"e75db8d7-1181-4cca-830d-569476a2a890\") " pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:01.164713 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.164550    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqmp\" (UniqueName: \"kubernetes.io/projected/e75db8d7-1181-4cca-830d-569476a2a890-kube-api-access-nzqmp\") pod \"must-gather-mqnbt\" (UID: \"e75db8d7-1181-4cca-830d-569476a2a890\") " pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:01.164953 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.164930    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e75db8d7-1181-4cca-830d-569476a2a890-must-gather-output\") pod \"must-gather-mqnbt\" (UID: \"e75db8d7-1181-4cca-830d-569476a2a890\") " pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:01.178321 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.178292    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqmp\" (UniqueName: \"kubernetes.io/projected/e75db8d7-1181-4cca-830d-569476a2a890-kube-api-access-nzqmp\") pod \"must-gather-mqnbt\" (UID: \"e75db8d7-1181-4cca-830d-569476a2a890\") " pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:01.249686 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.249650    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72p88/must-gather-mqnbt"
Sep 17 16:29:01.534313 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.534196    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72p88/must-gather-mqnbt"]
Sep 17 16:29:01.537173 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:29:01.537147    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode75db8d7_1181_4cca_830d_569476a2a890.slice/crio-1a360a2414398b38cb3af1caadc0a6fd7eda4c9d1b28b34925a15078bbd54a88 WatchSource:0}: Error finding container 1a360a2414398b38cb3af1caadc0a6fd7eda4c9d1b28b34925a15078bbd54a88: Status 404 returned error can't find the container with id 1a360a2414398b38cb3af1caadc0a6fd7eda4c9d1b28b34925a15078bbd54a88
Sep 17 16:29:01.539332 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:01.539314    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 16:29:02.289111 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:02.289077    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72p88/must-gather-mqnbt" event={"ID":"e75db8d7-1181-4cca-830d-569476a2a890","Type":"ContainerStarted","Data":"1a360a2414398b38cb3af1caadc0a6fd7eda4c9d1b28b34925a15078bbd54a88"}
Sep 17 16:29:03.295779 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:03.295743    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72p88/must-gather-mqnbt" event={"ID":"e75db8d7-1181-4cca-830d-569476a2a890","Type":"ContainerStarted","Data":"620dd1fad3909df7dc8146a7a910929ff6260f80cf70b2982f13ea968299f0b4"}
Sep 17 16:29:03.295779 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:03.295793    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72p88/must-gather-mqnbt" event={"ID":"e75db8d7-1181-4cca-830d-569476a2a890","Type":"ContainerStarted","Data":"b9efb9508a29b8118c15de862e7759df5b521f6af5da9818eb71ac5b6f0b8266"}
Sep 17 16:29:03.318728 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:03.318679    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-72p88/must-gather-mqnbt" podStartSLOduration=2.241228349 podStartE2EDuration="3.318661665s" totalImagesPullingTime="1.077433316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:29:03.317776677 +0000 UTC m=+2987.905821268" watchObservedRunningTime="2026-09-17 16:29:03.318661665 +0000 UTC m=+2987.906706251"
Sep 17 16:29:04.234091 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:04.234057    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9btfr_437c724d-ac60-4db7-861a-704df11b37c9/global-pull-secret-syncer/0.log"
Sep 17 16:29:04.453529 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:04.453491    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c5x5h_b69d31ab-249a-4aa8-b4bb-4245e9183d62/konnectivity-agent/0.log"
Sep 17 16:29:04.614139 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:04.614049    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-183.ec2.internal_78a56846a26782d79f6f1b96091c0b40/haproxy/0.log"
Sep 17 16:29:08.317269 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.317236    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e311e144-11f5-49b5-b4ff-49b5072889a6/alertmanager/0.log"
Sep 17 16:29:08.414673 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.414633    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e311e144-11f5-49b5-b4ff-49b5072889a6/config-reloader/0.log"
Sep 17 16:29:08.491225 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.491194    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e311e144-11f5-49b5-b4ff-49b5072889a6/kube-rbac-proxy-web/0.log"
Sep 17 16:29:08.554752 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.554596    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e311e144-11f5-49b5-b4ff-49b5072889a6/kube-rbac-proxy/0.log"
Sep 17 16:29:08.604979 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.604880    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e311e144-11f5-49b5-b4ff-49b5072889a6/kube-rbac-proxy-metric/0.log"
Sep 17 16:29:08.650689 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.650654    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e311e144-11f5-49b5-b4ff-49b5072889a6/prom-label-proxy/0.log"
Sep 17 16:29:08.697122 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.697087    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e311e144-11f5-49b5-b4ff-49b5072889a6/init-config-reloader/0.log"
Sep 17 16:29:08.756252 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.756217    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-4kbch_949248cb-2674-43d6-8127-31928befc780/cluster-monitoring-operator/0.log"
Sep 17 16:29:08.804743 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.804700    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-gkmrb_b870e596-5e4e-41ba-9c45-d694ab3d50a5/kube-state-metrics/0.log"
Sep 17 16:29:08.854133 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.854084    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-gkmrb_b870e596-5e4e-41ba-9c45-d694ab3d50a5/kube-rbac-proxy-main/0.log"
Sep 17 16:29:08.903252 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.903141    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-gkmrb_b870e596-5e4e-41ba-9c45-d694ab3d50a5/kube-rbac-proxy-self/0.log"
Sep 17 16:29:08.953667 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:08.953632    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85b6bfbb6d-wtwrn_6135a1b1-7a60-4d25-aecc-92a183de524f/metrics-server/0.log"
Sep 17 16:29:09.188642 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:09.188529    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwdf8_8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf/node-exporter/0.log"
Sep 17 16:29:09.233691 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:09.233655    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwdf8_8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf/kube-rbac-proxy/0.log"
Sep 17 16:29:09.274533 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:09.274500    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwdf8_8e8de3e9-4f0e-4da1-8955-99fce6ec9bcf/init-textfile/0.log"
Sep 17 16:29:09.422694 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:09.422663    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-hcvxd_ffa61525-735b-40bd-89ea-ccf07b290ab7/kube-rbac-proxy-main/0.log"
Sep 17 16:29:09.464649 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:09.464552    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-hcvxd_ffa61525-735b-40bd-89ea-ccf07b290ab7/kube-rbac-proxy-self/0.log"
Sep 17 16:29:09.503994 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:09.503960    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-hcvxd_ffa61525-735b-40bd-89ea-ccf07b290ab7/openshift-state-metrics/0.log"
Sep 17 16:29:10.105570 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:10.105405    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f75895595-jkq8h_5469f1f4-6bfb-48fc-a2cf-a606625648df/thanos-query/0.log"
Sep 17 16:29:10.142637 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:10.142577    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f75895595-jkq8h_5469f1f4-6bfb-48fc-a2cf-a606625648df/kube-rbac-proxy-web/0.log"
Sep 17 16:29:10.174602 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:10.174529    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f75895595-jkq8h_5469f1f4-6bfb-48fc-a2cf-a606625648df/kube-rbac-proxy/0.log"
Sep 17 16:29:10.216329 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:10.216295    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f75895595-jkq8h_5469f1f4-6bfb-48fc-a2cf-a606625648df/prom-label-proxy/0.log"
Sep 17 16:29:10.251117 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:10.251091    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f75895595-jkq8h_5469f1f4-6bfb-48fc-a2cf-a606625648df/kube-rbac-proxy-rules/0.log"
Sep 17 16:29:10.280414 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:10.280382    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f75895595-jkq8h_5469f1f4-6bfb-48fc-a2cf-a606625648df/kube-rbac-proxy-metrics/0.log"
Sep 17 16:29:11.373758 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:11.373728    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-snzd6_6bc73e9d-9065-4049-8079-59715503f460/networking-console-plugin/0.log"
Sep 17 16:29:11.871209 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:11.871173    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:29:11.876314 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:11.876284    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/3.log"
Sep 17 16:29:12.143886 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.143792    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"]
Sep 17 16:29:12.149473 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.149448    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.163069 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.163043    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"]
Sep 17 16:29:12.273585 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.273547    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-sys\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.273767 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.273612    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-proc\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.273767 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.273673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbnlk\" (UniqueName: \"kubernetes.io/projected/46dcd403-523f-458b-99b0-09e182aa8bca-kube-api-access-pbnlk\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.273767 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.273715    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-podres\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.273767 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.273742    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-lib-modules\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.356612 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.356582    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56c584c7cd-n9gxz_fd342800-5257-48b0-85ef-419922ac14be/console/0.log"
Sep 17 16:29:12.375205 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375163    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-sys\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375276    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-proc\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375288    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-sys\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375323    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbnlk\" (UniqueName: \"kubernetes.io/projected/46dcd403-523f-458b-99b0-09e182aa8bca-kube-api-access-pbnlk\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375345    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-podres\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375369    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-lib-modules\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375380    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-proc\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375509    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-lib-modules\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.375597 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.375545    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46dcd403-523f-458b-99b0-09e182aa8bca-podres\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.386588 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.386555    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbnlk\" (UniqueName: \"kubernetes.io/projected/46dcd403-523f-458b-99b0-09e182aa8bca-kube-api-access-pbnlk\") pod \"perf-node-gather-daemonset-gzx5p\" (UID: \"46dcd403-523f-458b-99b0-09e182aa8bca\") " pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.462216 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.462185    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:12.755537 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:12.755504    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"]
Sep 17 16:29:12.758781 ip-10-0-140-183 kubenswrapper[2568]: W0917 16:29:12.758704    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod46dcd403_523f_458b_99b0_09e182aa8bca.slice/crio-f18e60a5e018862c6c46eaf3aeb2860d905488a9078be183710abc5452e40b7b WatchSource:0}: Error finding container f18e60a5e018862c6c46eaf3aeb2860d905488a9078be183710abc5452e40b7b: Status 404 returned error can't find the container with id f18e60a5e018862c6c46eaf3aeb2860d905488a9078be183710abc5452e40b7b
Sep 17 16:29:13.363422 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:13.363375    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p" event={"ID":"46dcd403-523f-458b-99b0-09e182aa8bca","Type":"ContainerStarted","Data":"1d5cd3b6a37734085b2fa280ed0ad74cfac6aad0b6d3662a863206939b645946"}
Sep 17 16:29:13.363777 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:13.363445    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p" event={"ID":"46dcd403-523f-458b-99b0-09e182aa8bca","Type":"ContainerStarted","Data":"f18e60a5e018862c6c46eaf3aeb2860d905488a9078be183710abc5452e40b7b"}
Sep 17 16:29:13.364258 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:13.364230    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:14.048366 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:14.048337    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-llms2_5fc73331-de56-4a77-8391-bde40c51cae0/dns/0.log"
Sep 17 16:29:14.075243 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:14.075218    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-llms2_5fc73331-de56-4a77-8391-bde40c51cae0/kube-rbac-proxy/0.log"
Sep 17 16:29:14.214588 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:14.214549    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2k5tc_eda22602-f2c3-4cab-8e46-4835749a5b97/dns-node-resolver/0.log"
Sep 17 16:29:14.827404 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:14.827371    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6df7765dc8-85gxv_00e78b9c-d2cf-417a-a538-05623d7b1398/registry/0.log"
Sep 17 16:29:14.937385 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:14.937358    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kf4b5_1896bca3-c7f4-40bd-99c1-4a3f6d1a4365/node-ca/0.log"
Sep 17 16:29:16.037978 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:16.037889    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:29:16.056467 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:16.042857    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d8g8b_8e244896-d0b1-4caf-ae7b-a31ea72fad9c/console-operator/2.log"
Sep 17 16:29:16.265407 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:16.265376    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vsd8x_76cfc63e-e046-4228-aec2-8b884e290052/serve-healthcheck-canary/0.log"
Sep 17 16:29:16.948366 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:16.948339    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8264_31e273df-5c65-481f-b2e7-c7bd56604e47/kube-rbac-proxy/0.log"
Sep 17 16:29:16.977165 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:16.977132    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8264_31e273df-5c65-481f-b2e7-c7bd56604e47/exporter/0.log"
Sep 17 16:29:17.022556 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:17.022520    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8264_31e273df-5c65-481f-b2e7-c7bd56604e47/extractor/0.log"
Sep 17 16:29:19.533899 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:19.533859    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-74b7f9c78b-vpmlb_473bd3f5-cf1c-4ae1-8ea2-7f67c7640dda/manager/0.log"
Sep 17 16:29:19.694726 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:19.694691    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-bnthk_96d0667f-ce3b-4705-a7c4-5b5c763efc24/manager/0.log"
Sep 17 16:29:19.798467 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:19.798381    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-mtqfj_c97d432a-88ef-4a91-a2cf-1c369c5864a4/seaweedfs/0.log"
Sep 17 16:29:20.383967 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:20.383938    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p"
Sep 17 16:29:20.408913 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:20.408862    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-72p88/perf-node-gather-daemonset-gzx5p" podStartSLOduration=8.408831902 podStartE2EDuration="8.408831902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:29:13.387671909 +0000 UTC m=+2997.975716496" watchObservedRunningTime="2026-09-17 16:29:20.408831902 +0000 UTC m=+3004.996876489"
Sep 17 16:29:26.009823 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.009792    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql6ch_b8aadb40-1dda-4451-8765-ca03203e05a0/kube-multus-additional-cni-plugins/0.log"
Sep 17 16:29:26.043652 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.043623    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql6ch_b8aadb40-1dda-4451-8765-ca03203e05a0/egress-router-binary-copy/0.log"
Sep 17 16:29:26.072816 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.072791    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql6ch_b8aadb40-1dda-4451-8765-ca03203e05a0/cni-plugins/0.log"
Sep 17 16:29:26.098462 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.098437    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql6ch_b8aadb40-1dda-4451-8765-ca03203e05a0/bond-cni-plugin/0.log"
Sep 17 16:29:26.123938 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.123911    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql6ch_b8aadb40-1dda-4451-8765-ca03203e05a0/routeoverride-cni/0.log"
Sep 17 16:29:26.156558 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.156533    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql6ch_b8aadb40-1dda-4451-8765-ca03203e05a0/whereabouts-cni-bincopy/0.log"
Sep 17 16:29:26.185572 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.185543    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql6ch_b8aadb40-1dda-4451-8765-ca03203e05a0/whereabouts-cni/0.log"
Sep 17 16:29:26.431254 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.431163    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-htrfs_68386cc8-defe-4273-8d2e-2746d342d9d3/kube-multus/0.log"
Sep 17 16:29:26.536775 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.536748    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cgmvm_7ff164ba-34fa-45aa-b854-e48384d80808/network-metrics-daemon/0.log"
Sep 17 16:29:26.560583 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:26.560548    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cgmvm_7ff164ba-34fa-45aa-b854-e48384d80808/kube-rbac-proxy/0.log"
Sep 17 16:29:27.514052 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.514024    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/ovn-controller/0.log"
Sep 17 16:29:27.537052 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.537029    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/ovn-acl-logging/0.log"
Sep 17 16:29:27.565256 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.565232    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/kube-rbac-proxy-node/0.log"
Sep 17 16:29:27.591468 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.591451    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 16:29:27.614598 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.614575    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/northd/0.log"
Sep 17 16:29:27.641591 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.641570    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/nbdb/0.log"
Sep 17 16:29:27.675242 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.675218    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/sbdb/0.log"
Sep 17 16:29:27.729352 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:27.729328    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fm9p8_bd02d50f-d630-4da8-877c-dbb32eba205c/ovnkube-controller/0.log"
Sep 17 16:29:29.424055 ip-10-0-140-183 kubenswrapper[2568]: I0917 16:29:29.424014    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-b5xqc_a8098f77-1464-43c8-b25f-4991d2c44f90/network-check-target-container/0.log"