Sep 17 15:52:18.316994 ip-10-0-131-144 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 15:52:18.317006 ip-10-0-131-144 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 15:52:18.317016 ip-10-0-131-144 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 15:52:18.317332 ip-10-0-131-144 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 15:52:27.708279 ip-10-0-131-144 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 17 15:52:27.708292 ip-10-0-131-144 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8d222a1c4e5146a78eb7e8dc9cfc32dc --
Sep 17 15:54:10.437426 ip-10-0-131-144 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 15:54:10.874557 ip-10-0-131-144 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 15:54:10.874557 ip-10-0-131-144 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 15:54:10.874557 ip-10-0-131-144 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 15:54:10.874557 ip-10-0-131-144 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 15:54:10.874557 ip-10-0-131-144 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 15:54:10.877331 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.877246    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 15:54:10.882354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882340    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:54:10.882354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882354    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882358    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882363    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882368    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882371    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882374    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882377    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882380    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882383    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882386    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882389    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882391    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882394    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882397    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882399    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882402    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882406    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882409    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882412    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882414    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:54:10.882416 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882417    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882419    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882422    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882425    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882428    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882431    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882434    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882437    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882439    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882442    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882444    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882447    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882450    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882452    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882455    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882457    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882459    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882462    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882466    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:54:10.882889 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882470    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882474    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882477    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882480    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882482    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882485    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882488    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882490    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882493    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882495    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882498    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882502    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882506    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882510    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882514    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882538    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882557    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882562    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882566    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882570    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:54:10.883366 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882573    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882576    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882583    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882586    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882589    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882592    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882594    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882597    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882600    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882603    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882605    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882608    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882611    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882614    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882616    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882619    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882622    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882624    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882627    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:54:10.883867 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882629    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882632    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882635    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882638    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882640    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882643    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882646    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882649    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882651    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882656    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882659    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882662    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882665    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882668    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882670    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882673    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882677    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882680    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882682    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:54:10.884384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.882685    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:54:10.885296 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885283    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:54:10.885296 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885295    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885299    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885302    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885305    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885308    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885311    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885314    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885317    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885319    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885322    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885325    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885327    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885330    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885332    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885336    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885338    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885341    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885344    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885346    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885349    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:54:10.885354 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885351    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885354    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885357    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885360    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885363    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885366    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885369    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885371    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885374    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885376    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885379    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885382    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885384    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885387    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885389    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885394    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885397    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885400    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885402    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:54:10.885847 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885405    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885407    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885410    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885413    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885415    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885418    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885420    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885424    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885427    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885430    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885433    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885435    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885438    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885441    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885443    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885449    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885454    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885457    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885461    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:54:10.886420 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885465    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885468    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885471    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885474    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885476    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885479    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885482    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885485    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885488    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885491    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885494    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885496    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885499    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885501    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885504    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885506    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885509    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885511    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885514    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885517    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:54:10.886883 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885519    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885522    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885525    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885527    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885530    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885532    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885535    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885538    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885540    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885544    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885546    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885549    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885551    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885554    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885556    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885559    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885561    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885564    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885567    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885569    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:54:10.887384 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885923    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885927    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885930    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885933    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885938    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885941    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885945    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885948    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885951    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885954    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885957    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885959    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885962    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885965    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885968    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885970    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885973    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885976    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885978    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885981    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:54:10.887868 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885984    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885987    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885990    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885993    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885995    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.885997    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886000    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886002    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886005    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886008    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886011    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886013    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886016    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886019    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886021    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886024    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886026    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886029    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886031    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:54:10.888378 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886034    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886036    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886039    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886042    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886044    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886047    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886049    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886052    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886055    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886057    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886060    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886063    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886066    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886068    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886071    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886074    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886076    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886079    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886081    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886084    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:54:10.888836 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886086    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886089    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886093    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886097    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886100    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886103    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886106    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886109    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886112    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886114    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886117    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886120    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886122    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886125    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886130    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886132    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886136    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886140    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886143    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:54:10.889362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886146    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886148    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886151    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886166    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886169    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886172    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886175    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886177    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886180    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886183    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886186    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886189    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886191    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886194    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886197    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886199    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886202    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886204    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886207    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886209    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:54:10.889829 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886212    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886289    2566 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886320    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886326    2566 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886330    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886334    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886337    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886343    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886349    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886352    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886355    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886358    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886361    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886364    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886367    2566 flags.go:64] FLAG: --cgroup-root=""
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886370    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886373    2566 flags.go:64] FLAG: --client-ca-file=""
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886376    2566 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886379    2566 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886383    2566 flags.go:64] FLAG: --cluster-domain=""
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886386    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886389    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886392    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886396    2566 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 15:54:10.890338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886400    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886402    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886406    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886409    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886412    2566 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886415    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886418    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886421    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886424    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886428    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886431    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886434    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886437    2566 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886440    2566 flags.go:64] FLAG: --enable-server="true"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886443    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886448    2566 flags.go:64] FLAG: --event-burst="100"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886452    2566 flags.go:64] FLAG: --event-qps="50"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886455    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886458    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886461    2566 flags.go:64] FLAG: --eviction-hard=""
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886465    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886468    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886471    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886474    2566 flags.go:64] FLAG: --eviction-soft=""
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886477    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 15:54:10.890919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886480    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886483    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886486    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886489    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886491    2566 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886494    2566 flags.go:64] FLAG: --feature-gates=""
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886498    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886501    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886504    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886507    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886510    2566 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886513    2566 flags.go:64] FLAG: --help="false"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886516    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-131-144.ec2.internal"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886519    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886522    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886525    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886529    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886532    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886535    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886538    2566 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886541    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886544    2566 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886547    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886551    2566 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 15:54:10.891553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886554    2566 flags.go:64] FLAG: --kube-reserved=""
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886556    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886559    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886562    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886565    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886568    2566 flags.go:64] FLAG: --lock-file=""
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886571    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886573    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886576    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886586    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886589    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886592    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886595    2566 flags.go:64] FLAG: --logging-format="text"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886597    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886601    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886604    2566 flags.go:64] FLAG: --manifest-url=""
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886607    2566 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886611    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886614    2566 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886618    2566 flags.go:64] FLAG: --max-pods="110"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886621    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886623    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886626    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886629    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886632    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 15:54:10.892175 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886635    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886638    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886645    2566 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886647    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886650    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886653    2566 flags.go:64] FLAG: --pod-cidr=""
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886658    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886663    2566 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886665    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886668    2566 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886672    2566 flags.go:64] FLAG: --port="10250"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886674    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886677    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ac19e4d9be1be751"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886680    2566 flags.go:64] FLAG: --qos-reserved=""
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886683    2566 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886686    2566 flags.go:64] FLAG: --register-node="true"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886689    2566 flags.go:64] FLAG: --register-with-taints=""
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886692    2566 flags.go:64] FLAG: --registry-burst="10"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886695    2566 flags.go:64] FLAG: --registry-qps="5"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886698    2566 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886700    2566 flags.go:64] FLAG: --reserved-memory=""
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886704    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886707    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886710    2566 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886713    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 15:54:10.892827 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886715    2566 flags.go:64] FLAG: --runonce="false"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886718    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886721    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886724    2566 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886726    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886729    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886732    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886735    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886738    2566 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886741    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886744    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886747    2566 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886750    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886754    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886757    2566 flags.go:64] FLAG: --system-cgroups=""
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886760    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886765    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886768    2566 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886771    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886775    2566 flags.go:64] FLAG: --tls-min-version=""
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886778    2566 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886780    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886783    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886786    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886789    2566 flags.go:64] FLAG: --v="2"
Sep 17 15:54:10.893450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886792    2566 flags.go:64] FLAG: --version="false"
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886796    2566 flags.go:64] FLAG: --vmodule=""
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886799    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.886803    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886910    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886913    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886917    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886919    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886922    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886926    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886929    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886932    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886939    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886941    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886944    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886947    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886949    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886952    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886955    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886957    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:54:10.894110 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886961    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886964    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886967    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886970    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886973    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886975    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886978    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886981    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886983    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886986    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886988    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886991    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886994    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886996    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.886999    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887002    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887004    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887007    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887010    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887013    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:54:10.894625 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887015    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887018    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887021    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887023    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887027    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887030    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887032    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887035    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887037    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887040    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887042    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887045    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887049    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887051    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887054    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887056    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887059    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887062    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887065    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887067    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887070    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:54:10.895123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887072    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887075    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887078    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887080    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887083    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887086    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887088    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887091    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887093    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887096    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887098    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887101    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887104    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887108    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887112    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887116    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887119    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887123    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887125    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:54:10.895654 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887128    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887131    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887134    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887137    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887141    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887143    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887146    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887149    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887170    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887173    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887176    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887179    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887182    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887184    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887187    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887190    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887193    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887197    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887200    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:54:10.896112 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887202    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887205    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887208    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.887210    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.887994    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.895554    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.895566    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.895583    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.895589    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895651    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895655    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895659    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895662    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895665    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:54:10.896643 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895668    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895670    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895673    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895676    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895679    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895681    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895686    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895695    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895698    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895701    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895703    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895706    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895709    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895711    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895714    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895717    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895720    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895722    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895725    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:54:10.896995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895727    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895730    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895732    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895735    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895738    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895741    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895744    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895747    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895749    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895752    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895754    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895757    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895759    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895762    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895764    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895767    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895770    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895772    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895774    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895777    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:54:10.897506 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895780    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895789    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895791    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895794    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895796    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895799    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895802    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895804    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895807    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895809    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895812    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895814    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895817    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895820    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895822    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895825    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895827    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895831    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895834    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895836    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:54:10.897994 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895838    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895841    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895843    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895846    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895849    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895851    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895854    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895857    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895859    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895861    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895864    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895866    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895869    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895871    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895879    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895882    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895885    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895887    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895891    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:54:10.898736 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895894    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895897    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895900    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895903    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895906    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895909    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895913    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895915    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895918    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895921    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895924    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895926    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895929    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895931    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895934    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:54:10.899503 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.895936    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.895941    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896094    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896099    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896102    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896105    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896108    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896111    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896114    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896117    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896120    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896123    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896126    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896135    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896138    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 15:54:10.899989 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896141    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896143    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896146    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896149    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896151    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896167    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896171    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896174    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896177    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896179    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896182    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896185    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896188    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896190    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896193    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896195    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896198    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896200    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896203    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896206    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 15:54:10.900447 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896208    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896210    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896213    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896215    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896218    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896221    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896224    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896229    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896232    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896236    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896239    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896249    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896252    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896254    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896257    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896260    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896263    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896266    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896268    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 15:54:10.901088 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896271    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896274    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896277    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896279    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896282    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896285    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896288    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896290    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896292    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896295    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896297    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896300    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896302    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896305    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896307    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896310    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896312    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896315    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896317    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896320    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 15:54:10.901591 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896323    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896325    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896328    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896331    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896333    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896342    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896344    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896347    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896349    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896352    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896355    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896363    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896365    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896368    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896370    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896373    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896375    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896378    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896381    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896383    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 15:54:10.902123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896385    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896388    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896390    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896393    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896395    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896399    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:10.896402    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.896407    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.896505    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.900598    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.901679    2566 server.go:1019] "Starting client certificate rotation"
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.901781    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 15:54:10.902703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.901812    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 15:54:10.904677 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.904661    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 15:54:10.905851 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.905830    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 15:54:10.920383 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.920365    2566 log.go:25] "Validated CRI v1 runtime API"
Sep 17 15:54:10.923103 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.923082    2566 log.go:25] "Validated CRI v1 image API"
Sep 17 15:54:10.923979 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.923964    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 15:54:10.928152 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.928133    2566 fs.go:135] Filesystem UUIDs: map[3a512fcf-622a-4a7a-8cf1-b6afbe2ee392:/dev/nvme0n1p4 78dd17ca-f9ff-4c06-9960-385e8eff1186:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 17 15:54:10.928225 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.928151    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 15:54:10.932177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.932146    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 15:54:10.934816 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.934706    2566 manager.go:216] Machine: {Timestamp:2026-09-17 15:54:10.9323642 +0000 UTC m=+0.369040969 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100449 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2793f8363ffaa0136f294284cd585d SystemUUID:ec2793f8-363f-faa0-136f-294284cd585d BootID:8d222a1c-4e51-46a7-8eb7-e8dc9cfc32dc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:5f:4f:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:5f:4f:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:cc:1f:48:90:91 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 15:54:10.934816 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.934811    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 15:54:10.934921 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.934883    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 15:54:10.938560 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.938529    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 15:54:10.938701 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.938563    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-144.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 15:54:10.939360 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.939350    2566 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 15:54:10.939391 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.939363    2566 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 15:54:10.939391 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.939375    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 15:54:10.939391 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.939389    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 15:54:10.939483 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.939452    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 15:54:10.944184 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944172    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 15:54:10.944303 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944295    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 15:54:10.944333 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944322    2566 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 15:54:10.944333 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944331    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 15:54:10.944382 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944345    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 15:54:10.944382 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944354    2566 kubelet.go:398] "Adding apiserver pod source"
Sep 17 15:54:10.944382 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944369    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 15:54:10.944729 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944719    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 15:54:10.944767 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.944742    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 15:54:10.947461 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.947438    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 15:54:10.950516 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.950501    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 15:54:10.950573 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.950531    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 15:54:10.951887 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951871    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951893    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951902    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951910    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951922    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951931    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951950    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951958    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951968    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951976    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951987    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 15:54:10.951989 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.951995    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 15:54:10.953581 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.953569    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 15:54:10.953645 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.953583    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 15:54:10.957582 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.957566    2566 server.go:1268] "Started kubelet"
Sep 17 15:54:10.957721 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.957659    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 15:54:10.957817 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.957651    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 15:54:10.957870 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.957834    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 15:54:10.958109 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.958087    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-144.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 15:54:10.958235 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.958178    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 15:54:10.958235 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.958211    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-144.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 15:54:10.958462 ip-10-0-131-144 systemd[1]: Started Kubernetes Kubelet.
Sep 17 15:54:10.959446 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.959356    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 15:54:10.959840 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.959823    2566 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 15:54:10.963185 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.963143    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 15:54:10.965641 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.964530    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-144.ec2.internal.18d626b2b7e3b4ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-144.ec2.internal,UID:ip-10-0-131-144.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-144.ec2.internal,},FirstTimestamp:2026-09-17 15:54:10.95753851 +0000 UTC m=+0.394215280,LastTimestamp:2026-09-17 15:54:10.95753851 +0000 UTC m=+0.394215280,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-144.ec2.internal,}"
Sep 17 15:54:10.969527 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.969510    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 15:54:10.970269 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.970254    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 15:54:10.970424 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.970402    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:10.971015 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.970996    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 15:54:10.971015 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971012    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 15:54:10.971152 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971030    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 15:54:10.971152 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971092    2566 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 15:54:10.971152 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971101    2566 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 15:54:10.971319 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971302    2566 factory.go:55] Registering systemd factory
Sep 17 15:54:10.971371 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971342    2566 factory.go:223] Registration of the systemd container factory successfully
Sep 17 15:54:10.971419 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971308    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5x2zs"
Sep 17 15:54:10.971564 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971551    2566 factory.go:153] Registering CRI-O factory
Sep 17 15:54:10.971625 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971567    2566 factory.go:223] Registration of the crio container factory successfully
Sep 17 15:54:10.971625 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971612    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 15:54:10.971721 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971651    2566 factory.go:103] Registering Raw factory
Sep 17 15:54:10.971764 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.971724    2566 manager.go:1196] Started watching for new ooms in manager
Sep 17 15:54:10.972209 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.972186    2566 manager.go:349] Starting recovery of all containers
Sep 17 15:54:10.973532 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.973511    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-144.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 15:54:10.973631 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.973608    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 15:54:10.984768 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.984749    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5x2zs"
Sep 17 15:54:10.985535 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.985523    2566 manager.go:354] Recovery completed
Sep 17 15:54:10.989435 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.989421    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:54:10.991069 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.991051    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:54:10.991133 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.991074    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:54:10.991133 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.991084    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:54:10.991788 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.991771    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 15:54:10.991788 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.991785    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 15:54:10.991926 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.991796    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:54:10.991926 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.991803    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 15:54:10.992700 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.992683    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:54:10.992764 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.992716    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:54:10.992764 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.992731    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:54:10.993576 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:10.993514    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-144.ec2.internal.18d626b2b9e34985  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-144.ec2.internal,UID:ip-10-0-131-144.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-144.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-144.ec2.internal,},FirstTimestamp:2026-09-17 15:54:10.991065477 +0000 UTC m=+0.427742248,LastTimestamp:2026-09-17 15:54:10.991065477 +0000 UTC m=+0.427742248,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-144.ec2.internal,}"
Sep 17 15:54:10.993816 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.993805    2566 policy_none.go:49] "None policy: Start"
Sep 17 15:54:10.993886 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.993822    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 15:54:10.993886 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.993834    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 15:54:10.995975 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:10.995869    2566 policy_none.go:47] "Start"
Sep 17 15:54:11.023870 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.023852    2566 manager.go:337] "Starting Device Plugin manager"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.023893    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.023905    2566 server.go:87] "Starting device plugin registration server"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.024211    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.024222    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.024409    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.024510    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.024529    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.024994    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.025060    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.037656    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.038798    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.038851    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.038873    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.038976    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 15:54:11.048966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.041284    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:54:11.124371 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.124338    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:54:11.125188 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.125111    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:54:11.125188 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.125137    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:54:11.125188 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.125150    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:54:11.125328 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.125200    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.137616 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.137600    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.137659 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.137620    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-144.ec2.internal\": node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.139811 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.139795    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal"]
Sep 17 15:54:11.139855 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.139850    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:54:11.142046 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.142024    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:54:11.142046 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.142048    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:54:11.142169 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.142057    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:54:11.144377 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.144365    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:54:11.144520 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.144507    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.144586 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.144534    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:54:11.145189 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.145172    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:54:11.145261 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.145200    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:54:11.145261 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.145209    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:54:11.145261 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.145181    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:54:11.145376 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.145286    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:54:11.145376 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.145328    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:54:11.147530 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.147517    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.147579 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.147553    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 15:54:11.149640 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.149627    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 15:54:11.149714 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.149649    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 15:54:11.149714 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.149667    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeHasSufficientPID"
Sep 17 15:54:11.151447 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.151434    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.170702 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.170685    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-144.ec2.internal\" not found" node="ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.173978 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.173963    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-144.ec2.internal\" not found" node="ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.251842 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.251817    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.272065 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.272048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c1ed2aab8eb75b0ce187c7e8a27004a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal\" (UID: \"c1ed2aab8eb75b0ce187c7e8a27004a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.272119 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.272071    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1ed2aab8eb75b0ce187c7e8a27004a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal\" (UID: \"c1ed2aab8eb75b0ce187c7e8a27004a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.272119 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.272088    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbe89b2e26a424e79e58e066d84f237c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-144.ec2.internal\" (UID: \"fbe89b2e26a424e79e58e066d84f237c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.352668 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.352648    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.373000 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.372982    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c1ed2aab8eb75b0ce187c7e8a27004a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal\" (UID: \"c1ed2aab8eb75b0ce187c7e8a27004a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.373055 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.373006    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1ed2aab8eb75b0ce187c7e8a27004a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal\" (UID: \"c1ed2aab8eb75b0ce187c7e8a27004a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.373055 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.373022    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbe89b2e26a424e79e58e066d84f237c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-144.ec2.internal\" (UID: \"fbe89b2e26a424e79e58e066d84f237c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.373116 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.373068    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbe89b2e26a424e79e58e066d84f237c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-144.ec2.internal\" (UID: \"fbe89b2e26a424e79e58e066d84f237c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.373116 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.373084    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c1ed2aab8eb75b0ce187c7e8a27004a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal\" (UID: \"c1ed2aab8eb75b0ce187c7e8a27004a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.373116 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.373096    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1ed2aab8eb75b0ce187c7e8a27004a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal\" (UID: \"c1ed2aab8eb75b0ce187c7e8a27004a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.453448 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.453391    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.475841 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.475824    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.478998 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.478982    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.554056 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.554020    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.654516 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.654482    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.755080 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.755018    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-144.ec2.internal\" not found"
Sep 17 15:54:11.844687 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.844665    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:54:11.870786 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.870767    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.884786 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.884768    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 15:54:11.886946 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.886931    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal"
Sep 17 15:54:11.899808 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.899787    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 15:54:11.902069 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.902055    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 15:54:11.902186 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.902157    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 15:54:11.902250 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.902220    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 15:54:11.945593 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.945571    2566 apiserver.go:52] "Watching apiserver"
Sep 17 15:54:11.968424 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.968397    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 15:54:11.968855 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.968807    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-vvjn8","openshift-network-diagnostics/network-check-target-lbkhx","openshift-network-operator/iptables-alerter-lqrw5","openshift-ovn-kubernetes/ovnkube-node-h25hc","kube-system/global-pull-secret-syncer-d4xzc","kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl","openshift-cluster-node-tuning-operator/tuned-g85tb","openshift-dns/node-resolver-pmbs8","openshift-image-registry/node-ca-jj572","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal","openshift-multus/multus-6hslz","kube-system/konnectivity-agent-n7ws2","openshift-multus/multus-additional-cni-plugins-bwzqr"]
Sep 17 15:54:11.970141 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.970125    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 15:54:11.971652 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.971634    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:11.971745 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.971725    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:11.974536 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.974516    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:11.974619 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.974585    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:11.976074 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.976055    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbph8\" (UniqueName: \"kubernetes.io/projected/26f8edd7-4332-49c1-8f85-00a63475c7ed-kube-api-access-fbph8\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:11.976126 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.976085    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:11.976834 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.976821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:11.979304 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.979285    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:11.980369 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.980346    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 15:54:11.980476 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.980349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:54:11.980476 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.980457    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 15:54:11.980697 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.980684    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-46b8l\""
Sep 17 15:54:11.981702 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.981689    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:11.981811 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:11.981794    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:11.982202 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.982183    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5vlgb\""
Sep 17 15:54:11.982295 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.982279    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 15:54:11.983276 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.982932    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 15:54:11.983276 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.982966    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 15:54:11.983276 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.983069    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 15:54:11.983276 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.983214    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 15:54:11.983842 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.983822    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 15:54:11.985266 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.985216    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:11.987971 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.987952    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 15:54:11.988211 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.988192    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 15:54:11.988315 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.988197    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:11.988405 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.988385    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 15:54:11.988694 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.988675    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 15:54:11.989120 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.989107    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kgcg\""
Sep 17 15:54:11.990718 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.990699    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:11.990963 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.990908    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 15:49:11 +0000 UTC" deadline="2028-06-14 11:54:18.257847806 +0000 UTC"
Sep 17 15:54:11.990963 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.990961    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15260h0m6.266889734s"
Sep 17 15:54:11.991763 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.991737    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:54:11.991857 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.991764    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 15:54:11.991857 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.991809    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-djljt\""
Sep 17 15:54:11.993067 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.993053    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:11.994080 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.994064    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 15:54:11.994152 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.994102    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 15:54:11.994384 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.994339    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wjkhn\""
Sep 17 15:54:11.995264 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.995247    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lv2l7\""
Sep 17 15:54:11.995344 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.995308    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 15:54:11.995518 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.995503    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 15:54:11.995679 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.995664    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 15:54:11.995751 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.995705    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.998024    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.998283    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.998494    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.999155    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.999189    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:11.999358    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rb7vs\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.000652    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.001006    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.001025    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v4lv2\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.001029    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.003079    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.003752    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.003889    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 15:54:12.004685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.004075    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f898t\""
Sep 17 15:54:12.027860 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.027833    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ed2aab8eb75b0ce187c7e8a27004a6.slice/crio-bd3fa66f940e25e2f43d63d8594c6f5c3028e7ae3c17c926cbc03139f745c626 WatchSource:0}: Error finding container bd3fa66f940e25e2f43d63d8594c6f5c3028e7ae3c17c926cbc03139f745c626: Status 404 returned error can't find the container with id bd3fa66f940e25e2f43d63d8594c6f5c3028e7ae3c17c926cbc03139f745c626
Sep 17 15:54:12.028188 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.028153    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe89b2e26a424e79e58e066d84f237c.slice/crio-d956ebd83ef9ae854015c87dd156fbc7e4e1eb78c94eacf7326f244beede7c42 WatchSource:0}: Error finding container d956ebd83ef9ae854015c87dd156fbc7e4e1eb78c94eacf7326f244beede7c42: Status 404 returned error can't find the container with id d956ebd83ef9ae854015c87dd156fbc7e4e1eb78c94eacf7326f244beede7c42
Sep 17 15:54:12.035031 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.035016    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 15:54:12.039249 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.039219    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dx8df"
Sep 17 15:54:12.043745 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.043706    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal" event={"ID":"fbe89b2e26a424e79e58e066d84f237c","Type":"ContainerStarted","Data":"d956ebd83ef9ae854015c87dd156fbc7e4e1eb78c94eacf7326f244beede7c42"}
Sep 17 15:54:12.045057 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.045038    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal" event={"ID":"c1ed2aab8eb75b0ce187c7e8a27004a6","Type":"ContainerStarted","Data":"bd3fa66f940e25e2f43d63d8594c6f5c3028e7ae3c17c926cbc03139f745c626"}
Sep 17 15:54:12.053367 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.053348    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dx8df"
Sep 17 15:54:12.071952 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.071931    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 15:54:12.076267 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc51a31a-7fdd-4dc9-85d0-a484829f79bf-agent-certs\") pod \"konnectivity-agent-n7ws2\" (UID: \"bc51a31a-7fdd-4dc9-85d0-a484829f79bf\") " pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.076361 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-cnibin\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.076361 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-lib-modules\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.076361 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076304    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-log-socket\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.076361 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076319    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-socket-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.076361 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e29d81f-e0e8-43f7-b035-ef4bc9510105-dbus\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.076533 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-host\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.076533 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-run-ovn-kubernetes\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.076533 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076418    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-cni-netd\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.076533 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2hn\" (UniqueName: \"kubernetes.io/projected/67289bb4-1805-4d07-8583-aae240420d82-kube-api-access-9m2hn\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.076533 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076468    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.076533 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076494    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.076533 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-system-cni-dir\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076536    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysconfig\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076565    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysctl-d\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076578    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-run\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076596    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-sys\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da826eb5-f188-4354-a96b-062bc39c53cd-tmp\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076655    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71cbb99d-076e-430c-92a1-a135f7c133b8-hosts-file\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076699    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71cbb99d-076e-430c-92a1-a135f7c133b8-tmp-dir\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076724    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbvzc\" (UniqueName: \"kubernetes.io/projected/71cbb99d-076e-430c-92a1-a135f7c133b8-kube-api-access-gbvzc\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076742    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076758    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/228d91b2-8567-46ec-a469-c967cb61117c-ovn-node-metrics-cert\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076773    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-registration-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-kubelet\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076819    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-systemd\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.076844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-host\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076871    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-serviceca\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076888    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c8nb\" (UniqueName: \"kubernetes.io/projected/20531a4e-3ef6-4648-b15e-540ffd02bcc8-kube-api-access-9c8nb\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.076892    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.076908    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-cni-bin\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.076979    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:54:12.576955906 +0000 UTC m=+2.013632676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077006    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-env-overrides\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sds27\" (UniqueName: \"kubernetes.io/projected/4cd102a9-401a-4aba-974f-361d4f32c6ed-kube-api-access-sds27\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077071    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-modprobe-d\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-systemd\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077119    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-var-lib-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-system-cni-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-socket-dir-parent\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077206    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-netns\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077231    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-slash\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077255    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-k8s-cni-cncf-io\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.077519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67289bb4-1805-4d07-8583-aae240420d82-multus-daemon-config\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc51a31a-7fdd-4dc9-85d0-a484829f79bf-konnectivity-ca\") pod \"konnectivity-agent-n7ws2\" (UID: \"bc51a31a-7fdd-4dc9-85d0-a484829f79bf\") " pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077361    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-kubernetes\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077382    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-kubelet\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-ovnkube-config\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077424    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077440    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-sys-fs\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077459    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-cni-bin\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077495    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysctl-conf\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-var-lib-kubelet\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077536    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-device-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077559    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-hostroot\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077579    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nnjd\" (UniqueName: \"kubernetes.io/projected/a4244309-a935-44d0-9fe3-3edb7feb081f-kube-api-access-7nnjd\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077605    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da826eb5-f188-4354-a96b-062bc39c53cd-etc-tuned\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.077982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077634    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njr2p\" (UniqueName: \"kubernetes.io/projected/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-kube-api-access-njr2p\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-systemd-units\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077686    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-run-netns\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077703    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e29d81f-e0e8-43f7-b035-ef4bc9510105-kubelet-config\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077717    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67289bb4-1805-4d07-8583-aae240420d82-cni-binary-copy\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-cni-multus\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbph8\" (UniqueName: \"kubernetes.io/projected/26f8edd7-4332-49c1-8f85-00a63475c7ed-kube-api-access-fbph8\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077803    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-cnibin\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-etc-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077921    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077949    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-conf-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077963    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-multus-certs\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077977    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-etc-kubernetes\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.077992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-ovnkube-script-lib\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.078489 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.078006    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdvst\" (UniqueName: \"kubernetes.io/projected/228d91b2-8567-46ec-a469-c967cb61117c-kube-api-access-cdvst\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.078966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.078020    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-cni-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.078966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.078033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gl5j\" (UniqueName: \"kubernetes.io/projected/da826eb5-f188-4354-a96b-062bc39c53cd-kube-api-access-2gl5j\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.078966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.078047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20531a4e-3ef6-4648-b15e-540ffd02bcc8-iptables-alerter-script\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.078966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.078068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20531a4e-3ef6-4648-b15e-540ffd02bcc8-host-slash\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.078966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.078085    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-ovn\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.078966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.078100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-node-log\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.093591 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.093567    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 15:54:12.097105 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.097075    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbph8\" (UniqueName: \"kubernetes.io/projected/26f8edd7-4332-49c1-8f85-00a63475c7ed-kube-api-access-fbph8\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:12.178865 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178840    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-cnibin\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.178938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-lib-modules\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.178938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-log-socket\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.178938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178903    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-socket-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.178938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178928    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e29d81f-e0e8-43f7-b035-ef4bc9510105-dbus\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-host\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-log-socket\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-run-ovn-kubernetes\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-cnibin\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.178997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-cni-netd\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179042    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-lib-modules\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179048    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-run-ovn-kubernetes\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179047    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-cni-netd\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179063    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e29d81f-e0e8-43f7-b035-ef4bc9510105-dbus\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179072    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2hn\" (UniqueName: \"kubernetes.io/projected/67289bb4-1805-4d07-8583-aae240420d82-kube-api-access-9m2hn\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179083    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-socket-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179052    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-host\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-system-cni-dir\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.179203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179192    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179215    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysconfig\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.179230    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179256    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-system-cni-dir\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179275    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysconfig\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.179296    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret podName:9e29d81f-e0e8-43f7-b035-ef4bc9510105 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:12.679276223 +0000 UTC m=+2.115952988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret") pod "global-pull-secret-syncer-d4xzc" (UID: "9e29d81f-e0e8-43f7-b035-ef4bc9510105") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysctl-d\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-run\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-sys\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179398    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da826eb5-f188-4354-a96b-062bc39c53cd-tmp\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179422    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71cbb99d-076e-430c-92a1-a135f7c133b8-hosts-file\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysctl-d\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179447    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71cbb99d-076e-430c-92a1-a135f7c133b8-tmp-dir\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179486    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbvzc\" (UniqueName: \"kubernetes.io/projected/71cbb99d-076e-430c-92a1-a135f7c133b8-kube-api-access-gbvzc\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179495    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-sys\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179514    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71cbb99d-076e-430c-92a1-a135f7c133b8-hosts-file\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-run\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.179933 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179524    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179584    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/228d91b2-8567-46ec-a469-c967cb61117c-ovn-node-metrics-cert\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179586    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179609    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-registration-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179636    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-kubelet\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-systemd\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179695    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-host\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179716    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-registration-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-kubelet\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-serviceca\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179726    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71cbb99d-076e-430c-92a1-a135f7c133b8-tmp-dir\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-host\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-systemd\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179777    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c8nb\" (UniqueName: \"kubernetes.io/projected/20531a4e-3ef6-4648-b15e-540ffd02bcc8-kube-api-access-9c8nb\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179828    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-cni-bin\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-env-overrides\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.180860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sds27\" (UniqueName: \"kubernetes.io/projected/4cd102a9-401a-4aba-974f-361d4f32c6ed-kube-api-access-sds27\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-cni-bin\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179913    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-modprobe-d\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-systemd\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-var-lib-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.179992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-system-cni-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-socket-dir-parent\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180058    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-netns\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180071    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-modprobe-d\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180087    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-slash\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180108    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-var-lib-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-k8s-cni-cncf-io\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180153    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-k8s-cni-cncf-io\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180159    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-system-cni-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180215    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-systemd\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-netns\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-serviceca\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180267    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-slash\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.181740 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180155    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67289bb4-1805-4d07-8583-aae240420d82-multus-daemon-config\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180289    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-socket-dir-parent\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc51a31a-7fdd-4dc9-85d0-a484829f79bf-konnectivity-ca\") pod \"konnectivity-agent-n7ws2\" (UID: \"bc51a31a-7fdd-4dc9-85d0-a484829f79bf\") " pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180336    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180330    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-env-overrides\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-kubernetes\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-kubelet\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180463    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-ovnkube-config\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180506    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-kubernetes\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180515    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-sys-fs\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180551    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-kubelet\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180570    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-cni-bin\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180585    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180597    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180610    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-sys-fs\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.182425 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180624    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysctl-conf\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180632    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-cni-bin\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180649    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-var-lib-kubelet\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180674    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-device-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-hostroot\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180708    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67289bb4-1805-4d07-8583-aae240420d82-multus-daemon-config\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nnjd\" (UniqueName: \"kubernetes.io/projected/a4244309-a935-44d0-9fe3-3edb7feb081f-kube-api-access-7nnjd\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180754    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da826eb5-f188-4354-a96b-062bc39c53cd-etc-tuned\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-etc-sysctl-conf\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-device-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180780    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njr2p\" (UniqueName: \"kubernetes.io/projected/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-kube-api-access-njr2p\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180808    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-systemd-units\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180823    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da826eb5-f188-4354-a96b-062bc39c53cd-var-lib-kubelet\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-run-netns\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e29d81f-e0e8-43f7-b035-ef4bc9510105-kubelet-config\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-hostroot\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180909    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183108 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180935    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67289bb4-1805-4d07-8583-aae240420d82-cni-binary-copy\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180963    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-cni-multus\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.180993    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181019    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-cnibin\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181029    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-ovnkube-config\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181048    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-etc-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181082    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181101    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-conf-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181122    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-host-run-netns\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-multus-certs\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181139    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4244309-a935-44d0-9fe3-3edb7feb081f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181182    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e29d81f-e0e8-43f7-b035-ef4bc9510105-kubelet-config\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181210    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4244309-a935-44d0-9fe3-3edb7feb081f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181225    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-cnibin\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181229    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-var-lib-cni-multus\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181238    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-etc-kubernetes\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181268    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-systemd-units\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.183707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181273    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-etc-kubernetes\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181292    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-ovnkube-script-lib\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181322    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-etc-openvswitch\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181323    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cd102a9-401a-4aba-974f-361d4f32c6ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181351    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-host-run-multus-certs\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181353    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdvst\" (UniqueName: \"kubernetes.io/projected/228d91b2-8567-46ec-a469-c967cb61117c-kube-api-access-cdvst\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-cni-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-conf-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gl5j\" (UniqueName: \"kubernetes.io/projected/da826eb5-f188-4354-a96b-062bc39c53cd-kube-api-access-2gl5j\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181450    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20531a4e-3ef6-4648-b15e-540ffd02bcc8-iptables-alerter-script\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181448    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67289bb4-1805-4d07-8583-aae240420d82-multus-cni-dir\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181478    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20531a4e-3ef6-4648-b15e-540ffd02bcc8-host-slash\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-ovn\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181541    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-node-log\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc51a31a-7fdd-4dc9-85d0-a484829f79bf-agent-certs\") pod \"konnectivity-agent-n7ws2\" (UID: \"bc51a31a-7fdd-4dc9-85d0-a484829f79bf\") " pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181602    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20531a4e-3ef6-4648-b15e-540ffd02bcc8-host-slash\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-node-log\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67289bb4-1805-4d07-8583-aae240420d82-cni-binary-copy\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.184252 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181704    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/228d91b2-8567-46ec-a469-c967cb61117c-run-ovn\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184771 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc51a31a-7fdd-4dc9-85d0-a484829f79bf-konnectivity-ca\") pod \"konnectivity-agent-n7ws2\" (UID: \"bc51a31a-7fdd-4dc9-85d0-a484829f79bf\") " pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.184771 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.181959    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/228d91b2-8567-46ec-a469-c967cb61117c-ovnkube-script-lib\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184771 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.182099    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20531a4e-3ef6-4648-b15e-540ffd02bcc8-iptables-alerter-script\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.184771 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.182659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/228d91b2-8567-46ec-a469-c967cb61117c-ovn-node-metrics-cert\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.184771 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.182833    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da826eb5-f188-4354-a96b-062bc39c53cd-tmp\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.184771 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.183679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da826eb5-f188-4354-a96b-062bc39c53cd-etc-tuned\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.184771 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.184277    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc51a31a-7fdd-4dc9-85d0-a484829f79bf-agent-certs\") pod \"konnectivity-agent-n7ws2\" (UID: \"bc51a31a-7fdd-4dc9-85d0-a484829f79bf\") " pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.191702 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.191677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2hn\" (UniqueName: \"kubernetes.io/projected/67289bb4-1805-4d07-8583-aae240420d82-kube-api-access-9m2hn\") pod \"multus-6hslz\" (UID: \"67289bb4-1805-4d07-8583-aae240420d82\") " pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.191943 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.191926    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbvzc\" (UniqueName: \"kubernetes.io/projected/71cbb99d-076e-430c-92a1-a135f7c133b8-kube-api-access-gbvzc\") pod \"node-resolver-pmbs8\" (UID: \"71cbb99d-076e-430c-92a1-a135f7c133b8\") " pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.192212 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.192196    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c8nb\" (UniqueName: \"kubernetes.io/projected/20531a4e-3ef6-4648-b15e-540ffd02bcc8-kube-api-access-9c8nb\") pod \"iptables-alerter-lqrw5\" (UID: \"20531a4e-3ef6-4648-b15e-540ffd02bcc8\") " pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.205098 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.205077    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:54:12.205192 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.205106    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:54:12.205192 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.205122    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5t4qj for pod openshift-network-diagnostics/network-check-target-lbkhx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:12.205286 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.205202    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj podName:5152d327-3905-4130-bc08-0e23cf39a6b2 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:12.705182554 +0000 UTC m=+2.141859324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5t4qj" (UniqueName: "kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj") pod "network-check-target-lbkhx" (UID: "5152d327-3905-4130-bc08-0e23cf39a6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:12.207074 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.207052    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sds27\" (UniqueName: \"kubernetes.io/projected/4cd102a9-401a-4aba-974f-361d4f32c6ed-kube-api-access-sds27\") pod \"aws-ebs-csi-driver-node-s77hl\" (UID: \"4cd102a9-401a-4aba-974f-361d4f32c6ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.207074 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.207071    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njr2p\" (UniqueName: \"kubernetes.io/projected/4c789c20-2a1f-4b93-8b12-d9c8144c53a7-kube-api-access-njr2p\") pod \"node-ca-jj572\" (UID: \"4c789c20-2a1f-4b93-8b12-d9c8144c53a7\") " pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.207352 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.207332    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gl5j\" (UniqueName: \"kubernetes.io/projected/da826eb5-f188-4354-a96b-062bc39c53cd-kube-api-access-2gl5j\") pod \"tuned-g85tb\" (UID: \"da826eb5-f188-4354-a96b-062bc39c53cd\") " pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.209938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.209917    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nnjd\" (UniqueName: \"kubernetes.io/projected/a4244309-a935-44d0-9fe3-3edb7feb081f-kube-api-access-7nnjd\") pod \"multus-additional-cni-plugins-bwzqr\" (UID: \"a4244309-a935-44d0-9fe3-3edb7feb081f\") " pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.211474 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.211458    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdvst\" (UniqueName: \"kubernetes.io/projected/228d91b2-8567-46ec-a469-c967cb61117c-kube-api-access-cdvst\") pod \"ovnkube-node-h25hc\" (UID: \"228d91b2-8567-46ec-a469-c967cb61117c\") " pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.310030 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.309950    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqrw5"
Sep 17 15:54:12.320562 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.320540    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20531a4e_3ef6_4648_b15e_540ffd02bcc8.slice/crio-db164f0a096f0bdc894520f884690a9065b090b99491e1ec7c528b951bd0bc46 WatchSource:0}: Error finding container db164f0a096f0bdc894520f884690a9065b090b99491e1ec7c528b951bd0bc46: Status 404 returned error can't find the container with id db164f0a096f0bdc894520f884690a9065b090b99491e1ec7c528b951bd0bc46
Sep 17 15:54:12.327022 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.327005    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:12.335042 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.335014    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod228d91b2_8567_46ec_a469_c967cb61117c.slice/crio-9095c2c61589e86bd0f2cff7bf755eb69f40e4cb3dbb6bb081541aa0b27789e8 WatchSource:0}: Error finding container 9095c2c61589e86bd0f2cff7bf755eb69f40e4cb3dbb6bb081541aa0b27789e8: Status 404 returned error can't find the container with id 9095c2c61589e86bd0f2cff7bf755eb69f40e4cb3dbb6bb081541aa0b27789e8
Sep 17 15:54:12.341566 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.341547    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl"
Sep 17 15:54:12.345573 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.345551    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g85tb"
Sep 17 15:54:12.351757 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.351610    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pmbs8"
Sep 17 15:54:12.352445 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.352101    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd102a9_401a_4aba_974f_361d4f32c6ed.slice/crio-d5576ed25af90cb17136d6ff3867e25c40163112e90c9e8fa85a13ce70694381 WatchSource:0}: Error finding container d5576ed25af90cb17136d6ff3867e25c40163112e90c9e8fa85a13ce70694381: Status 404 returned error can't find the container with id d5576ed25af90cb17136d6ff3867e25c40163112e90c9e8fa85a13ce70694381
Sep 17 15:54:12.357725 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.357704    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jj572"
Sep 17 15:54:12.357995 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.357971    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda826eb5_f188_4354_a96b_062bc39c53cd.slice/crio-562c0ebd0447abba3c3c521537d9e73f8eec3a7adf9b08d9a71a33603b45d7ca WatchSource:0}: Error finding container 562c0ebd0447abba3c3c521537d9e73f8eec3a7adf9b08d9a71a33603b45d7ca: Status 404 returned error can't find the container with id 562c0ebd0447abba3c3c521537d9e73f8eec3a7adf9b08d9a71a33603b45d7ca
Sep 17 15:54:12.361489 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.361422    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cbb99d_076e_430c_92a1_a135f7c133b8.slice/crio-6ca513e73197a9f99fb1b731b642e715389e7ef39739cdb9c21443c6ca42a848 WatchSource:0}: Error finding container 6ca513e73197a9f99fb1b731b642e715389e7ef39739cdb9c21443c6ca42a848: Status 404 returned error can't find the container with id 6ca513e73197a9f99fb1b731b642e715389e7ef39739cdb9c21443c6ca42a848
Sep 17 15:54:12.363731 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.363712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6hslz"
Sep 17 15:54:12.366661 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.366623    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c789c20_2a1f_4b93_8b12_d9c8144c53a7.slice/crio-9f1adf982d07e6bbc179d73301b18a923a5e64367d3934ab95a2bb37e8469172 WatchSource:0}: Error finding container 9f1adf982d07e6bbc179d73301b18a923a5e64367d3934ab95a2bb37e8469172: Status 404 returned error can't find the container with id 9f1adf982d07e6bbc179d73301b18a923a5e64367d3934ab95a2bb37e8469172
Sep 17 15:54:12.369326 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.369309    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:12.372118 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.372096    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67289bb4_1805_4d07_8583_aae240420d82.slice/crio-b8137bd744a467edb95551bc7419af25010a1465f3b496f9ec228349bffda670 WatchSource:0}: Error finding container b8137bd744a467edb95551bc7419af25010a1465f3b496f9ec228349bffda670: Status 404 returned error can't find the container with id b8137bd744a467edb95551bc7419af25010a1465f3b496f9ec228349bffda670
Sep 17 15:54:12.374378 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.374358    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bwzqr"
Sep 17 15:54:12.379399 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.379377    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc51a31a_7fdd_4dc9_85d0_a484829f79bf.slice/crio-d38175440de07b4423fb1d7dbd7bd37956f00552a4b1fc5689a39beb3d0fb630 WatchSource:0}: Error finding container d38175440de07b4423fb1d7dbd7bd37956f00552a4b1fc5689a39beb3d0fb630: Status 404 returned error can't find the container with id d38175440de07b4423fb1d7dbd7bd37956f00552a4b1fc5689a39beb3d0fb630
Sep 17 15:54:12.384362 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:12.384336    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4244309_a935_44d0_9fe3_3edb7feb081f.slice/crio-572214fb4069c1944325e0de891d74a1c140a52243ee04d397a51c8f4ffb240f WatchSource:0}: Error finding container 572214fb4069c1944325e0de891d74a1c140a52243ee04d397a51c8f4ffb240f: Status 404 returned error can't find the container with id 572214fb4069c1944325e0de891d74a1c140a52243ee04d397a51c8f4ffb240f
Sep 17 15:54:12.466295 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.466264    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:54:12.584844 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.584762    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:12.585002 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.584927    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:12.585002 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.584987    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:54:13.584967497 +0000 UTC m=+3.021644261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:12.685873 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.685838    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:12.686065 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.685990    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:12.686065 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.686057    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret podName:9e29d81f-e0e8-43f7-b035-ef4bc9510105 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:13.686038423 +0000 UTC m=+3.122715181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret") pod "global-pull-secret-syncer-d4xzc" (UID: "9e29d81f-e0e8-43f7-b035-ef4bc9510105") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:12.786557 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.786512    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:12.786737 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.786724    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:54:12.786806 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.786748    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:54:12.786806 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.786762    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5t4qj for pod openshift-network-diagnostics/network-check-target-lbkhx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:12.786910 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:12.786829    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj podName:5152d327-3905-4130-bc08-0e23cf39a6b2 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:13.786808633 +0000 UTC m=+3.223485410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5t4qj" (UniqueName: "kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj") pod "network-check-target-lbkhx" (UID: "5152d327-3905-4130-bc08-0e23cf39a6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:12.842538 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:12.842267    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:54:13.057657 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.057536    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 15:49:12 +0000 UTC" deadline="2028-03-22 02:59:27.142026476 +0000 UTC"
Sep 17 15:54:13.057657 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.057596    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13235h5m14.08443562s"
Sep 17 15:54:13.059071 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.059021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g85tb" event={"ID":"da826eb5-f188-4354-a96b-062bc39c53cd","Type":"ContainerStarted","Data":"562c0ebd0447abba3c3c521537d9e73f8eec3a7adf9b08d9a71a33603b45d7ca"}
Sep 17 15:54:13.080594 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.080513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqrw5" event={"ID":"20531a4e-3ef6-4648-b15e-540ffd02bcc8","Type":"ContainerStarted","Data":"db164f0a096f0bdc894520f884690a9065b090b99491e1ec7c528b951bd0bc46"}
Sep 17 15:54:13.089181 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.089146    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerStarted","Data":"572214fb4069c1944325e0de891d74a1c140a52243ee04d397a51c8f4ffb240f"}
Sep 17 15:54:13.103300 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.103213    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jj572" event={"ID":"4c789c20-2a1f-4b93-8b12-d9c8144c53a7","Type":"ContainerStarted","Data":"9f1adf982d07e6bbc179d73301b18a923a5e64367d3934ab95a2bb37e8469172"}
Sep 17 15:54:13.105651 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.105623    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl" event={"ID":"4cd102a9-401a-4aba-974f-361d4f32c6ed","Type":"ContainerStarted","Data":"d5576ed25af90cb17136d6ff3867e25c40163112e90c9e8fa85a13ce70694381"}
Sep 17 15:54:13.124090 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.124041    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"9095c2c61589e86bd0f2cff7bf755eb69f40e4cb3dbb6bb081541aa0b27789e8"}
Sep 17 15:54:13.145427 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.145397    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n7ws2" event={"ID":"bc51a31a-7fdd-4dc9-85d0-a484829f79bf","Type":"ContainerStarted","Data":"d38175440de07b4423fb1d7dbd7bd37956f00552a4b1fc5689a39beb3d0fb630"}
Sep 17 15:54:13.165228 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.165201    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6hslz" event={"ID":"67289bb4-1805-4d07-8583-aae240420d82","Type":"ContainerStarted","Data":"b8137bd744a467edb95551bc7419af25010a1465f3b496f9ec228349bffda670"}
Sep 17 15:54:13.195337 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.195272    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pmbs8" event={"ID":"71cbb99d-076e-430c-92a1-a135f7c133b8","Type":"ContainerStarted","Data":"6ca513e73197a9f99fb1b731b642e715389e7ef39739cdb9c21443c6ca42a848"}
Sep 17 15:54:13.593216 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.593126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:13.593397 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.593330    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:13.593462 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.593399    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:54:15.593378844 +0000 UTC m=+5.030055624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:13.694621 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.693994    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:13.694621 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.694157    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:13.694621 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.694240    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret podName:9e29d81f-e0e8-43f7-b035-ef4bc9510105 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:15.694202338 +0000 UTC m=+5.130879120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret") pod "global-pull-secret-syncer-d4xzc" (UID: "9e29d81f-e0e8-43f7-b035-ef4bc9510105") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:13.795255 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:13.794599    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:13.795255 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.794763    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:54:13.795255 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.794793    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:54:13.795255 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.794807    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5t4qj for pod openshift-network-diagnostics/network-check-target-lbkhx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:13.795255 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:13.794878    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj podName:5152d327-3905-4130-bc08-0e23cf39a6b2 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:15.794857786 +0000 UTC m=+5.231534576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5t4qj" (UniqueName: "kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj") pod "network-check-target-lbkhx" (UID: "5152d327-3905-4130-bc08-0e23cf39a6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:14.040305 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:14.039351    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:14.040305 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:14.039488    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:14.040305 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:14.039889    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:14.040305 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:14.040004    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:14.040305 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:14.040072    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:14.040305 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:14.040142    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:14.058739 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:14.058668    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 15:49:12 +0000 UTC" deadline="2028-03-23 12:49:00.897375959 +0000 UTC"
Sep 17 15:54:14.058739 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:14.058711    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13268h54m46.838670298s"
Sep 17 15:54:14.837904 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:14.837872    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 15:54:15.609573 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:15.609535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:15.610072 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.609706    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:15.610072 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.609772    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:54:19.609751421 +0000 UTC m=+9.046428186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:15.710518 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:15.710471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:15.710699 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.710650    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:15.710772 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.710711    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret podName:9e29d81f-e0e8-43f7-b035-ef4bc9510105 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:19.71069168 +0000 UTC m=+9.147368462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret") pod "global-pull-secret-syncer-d4xzc" (UID: "9e29d81f-e0e8-43f7-b035-ef4bc9510105") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:15.812317 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:15.811634    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:15.812317 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.811847    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:54:15.812317 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.811871    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:54:15.812317 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.811883    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5t4qj for pod openshift-network-diagnostics/network-check-target-lbkhx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:15.812317 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:15.811942    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj podName:5152d327-3905-4130-bc08-0e23cf39a6b2 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:19.811922228 +0000 UTC m=+9.248599002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5t4qj" (UniqueName: "kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj") pod "network-check-target-lbkhx" (UID: "5152d327-3905-4130-bc08-0e23cf39a6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:16.039891 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:16.039644    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:16.040052 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:16.040009    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:16.040480 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:16.040253    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:16.040480 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:16.040276    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:16.040480 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:16.040405    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:16.040694 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:16.040496    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:18.039392 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:18.039354    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:18.039878 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:18.039399    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:18.039878 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:18.039506    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:18.039878 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:18.039764    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:18.040065 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:18.039943    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:18.040121 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:18.040055    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:19.646202 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:19.646147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:19.646706 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.646320    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:19.646706 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.646394    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:54:27.646371909 +0000 UTC m=+17.083048670 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:19.747407 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:19.747365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:19.747680 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.747655    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:19.747770 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.747756    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret podName:9e29d81f-e0e8-43f7-b035-ef4bc9510105 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:27.747724749 +0000 UTC m=+17.184401510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret") pod "global-pull-secret-syncer-d4xzc" (UID: "9e29d81f-e0e8-43f7-b035-ef4bc9510105") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:19.848592 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:19.848546    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:19.848762 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.848729    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:54:19.848762 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.848755    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:54:19.848898 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.848769    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5t4qj for pod openshift-network-diagnostics/network-check-target-lbkhx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:19.848898 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:19.848837    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj podName:5152d327-3905-4130-bc08-0e23cf39a6b2 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:27.848817923 +0000 UTC m=+17.285494685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5t4qj" (UniqueName: "kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj") pod "network-check-target-lbkhx" (UID: "5152d327-3905-4130-bc08-0e23cf39a6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:20.040081 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:20.039509    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:20.040081 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:20.039549    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:20.040081 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:20.039655    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:20.040366 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:20.040095    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:20.040366 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:20.040150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:20.040366 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:20.040242    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:22.039859 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:22.039825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:22.040399 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:22.039867    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:22.040399 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:22.039823    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:22.040399 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:22.039958    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:22.040399 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:22.040063    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:22.040399 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:22.040154    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:24.039256 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:24.039215    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:24.039256 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:24.039240    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:24.039767 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:24.039212    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:24.039767 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:24.039341    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:24.039767 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:24.039425    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:24.039767 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:24.039511    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:26.039810 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:26.039778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:26.039810 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:26.039804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:26.040324 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:26.039804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:26.040324 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:26.039915    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:26.040324 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:26.040032    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:26.040324 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:26.040105    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:27.707378 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:27.707312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:27.707799 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.707468    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:27.707799 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.707542    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:54:43.70752167 +0000 UTC m=+33.144198450 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:27.808283 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:27.808253    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:27.808446 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.808423    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:27.808518 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.808506    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret podName:9e29d81f-e0e8-43f7-b035-ef4bc9510105 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:43.80848587 +0000 UTC m=+33.245162644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret") pod "global-pull-secret-syncer-d4xzc" (UID: "9e29d81f-e0e8-43f7-b035-ef4bc9510105") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:27.909243 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:27.909212    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:27.909382 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.909340    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:54:27.909382 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.909356    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:54:27.909382 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.909364    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5t4qj for pod openshift-network-diagnostics/network-check-target-lbkhx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:27.909500 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:27.909413    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj podName:5152d327-3905-4130-bc08-0e23cf39a6b2 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:43.909401607 +0000 UTC m=+33.346078364 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5t4qj" (UniqueName: "kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj") pod "network-check-target-lbkhx" (UID: "5152d327-3905-4130-bc08-0e23cf39a6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:28.039915 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:28.039858    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:28.040014 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:28.039858    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:28.040063 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:28.040023    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:28.040063 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:28.039945    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:28.040063 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:28.039858    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:28.040184 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:28.040101    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:30.039442 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:30.039405    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:30.039910 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:30.039405    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:30.039910 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:30.039536    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:30.039910 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:30.039412    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:30.039910 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:30.039649    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:30.039910 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:30.039759    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:32.039381 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:32.039356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:32.039725 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:32.039383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:32.039725 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:32.039396    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:32.039725 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:32.039470    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:32.039725 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:32.039553    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:32.039725 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:32.039635    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:32.240667 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:32.240637    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g85tb" event={"ID":"da826eb5-f188-4354-a96b-062bc39c53cd","Type":"ContainerStarted","Data":"35ea91f1e91ac15e2f305ebf29bdc2ebb849331cfc486e2aa593720425522508"}
Sep 17 15:54:32.245271 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:32.245246    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal" event={"ID":"fbe89b2e26a424e79e58e066d84f237c","Type":"ContainerStarted","Data":"c7200ae975764b87663cfb6aaf875ba2ba432443e769b0594f3c0663cb1e6afa"}
Sep 17 15:54:32.287610 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:32.287454    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g85tb" podStartSLOduration=0.616245751 podStartE2EDuration="20.287441349s" totalImagesPullingTime="19.671195598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:32.287360036 +0000 UTC m=+21.724036816" watchObservedRunningTime="2026-09-17 15:54:32.287441349 +0000 UTC m=+21.724118127"
Sep 17 15:54:33.249023 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.248791    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jj572" event={"ID":"4c789c20-2a1f-4b93-8b12-d9c8144c53a7","Type":"ContainerStarted","Data":"3fde443cdcf1a2024e5db595367c61dadc5e0246f04a7264d8de0d5904440a04"}
Sep 17 15:54:33.250437 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.250416    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl" event={"ID":"4cd102a9-401a-4aba-974f-361d4f32c6ed","Type":"ContainerStarted","Data":"62b69c78c3145f9665a6c70e5ba1f62cad31f8cb7ad819f31e3596bc7470e05b"}
Sep 17 15:54:33.255810 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.253505    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 15:54:33.256255 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.256229    2566 generic.go:356] "Generic (PLEG): container finished" podID="228d91b2-8567-46ec-a469-c967cb61117c" containerID="1c8c5aa893f3c51088bfc32534cae6c1214e76b7644901b581a305705fb7a673" exitCode=1
Sep 17 15:54:33.256331 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.256300    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"a9a213029535fc5e3463568c3e3676df40bbac5f056d4a38d00cc7b1ae3fc953"}
Sep 17 15:54:33.256424 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.256406    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"c3ebad7865d969ba1567a43461518d11144437948bfd8435e2ac1a36db0f84a6"}
Sep 17 15:54:33.256475 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.256462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"87c570c0cb44cdb1d5d56c94831ac5d17cba26d3a9feb465f4b5e289544e1984"}
Sep 17 15:54:33.256527 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.256514    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"502f8e93f9fc50a2a43d4c3f254216ca72d04b64a094b9a38d415da0df5a0646"}
Sep 17 15:54:33.256584 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.256550    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerDied","Data":"1c8c5aa893f3c51088bfc32534cae6c1214e76b7644901b581a305705fb7a673"}
Sep 17 15:54:33.256584 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.256581    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"d89536d9652162f113d4b4f17c17ba4bfc243691d1d98553326c7281df0ad4ca"}
Sep 17 15:54:33.257748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.257710    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n7ws2" event={"ID":"bc51a31a-7fdd-4dc9-85d0-a484829f79bf","Type":"ContainerStarted","Data":"60bcf7f98d78e620da68c9a62623e8a6660efd2c47a925c8e43462cee077739c"}
Sep 17 15:54:33.259071 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.259048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6hslz" event={"ID":"67289bb4-1805-4d07-8583-aae240420d82","Type":"ContainerStarted","Data":"a1d5385133d9314670839dacd665f1c1c77337bd9d5915e5a422d950af47ad26"}
Sep 17 15:54:33.260383 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.260361    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pmbs8" event={"ID":"71cbb99d-076e-430c-92a1-a135f7c133b8","Type":"ContainerStarted","Data":"dc23e6ef61c2bec47e7e564496641ef23fd7e02b27e465918eea555ab2c38938"}
Sep 17 15:54:33.261630 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.261607    2566 generic.go:356] "Generic (PLEG): container finished" podID="c1ed2aab8eb75b0ce187c7e8a27004a6" containerID="1a3224670cbf050fd8b89d80d853bfe91b3f7422bbf2aedcf4ecf4fe2c9ecd45" exitCode=0
Sep 17 15:54:33.261710 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.261632    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal" event={"ID":"c1ed2aab8eb75b0ce187c7e8a27004a6","Type":"ContainerDied","Data":"1a3224670cbf050fd8b89d80d853bfe91b3f7422bbf2aedcf4ecf4fe2c9ecd45"}
Sep 17 15:54:33.263117 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.263095    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4244309-a935-44d0-9fe3-3edb7feb081f" containerID="af24037dcc93c85a9932dc84b4aeba93725fc7b427cd272fd4b9d935ed1d4df4" exitCode=0
Sep 17 15:54:33.263232 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.263181    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerDied","Data":"af24037dcc93c85a9932dc84b4aeba93725fc7b427cd272fd4b9d935ed1d4df4"}
Sep 17 15:54:33.271630 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.271598    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-144.ec2.internal" podStartSLOduration=21.27158731 podStartE2EDuration="21.27158731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:32.331077722 +0000 UTC m=+21.767754506" watchObservedRunningTime="2026-09-17 15:54:33.27158731 +0000 UTC m=+22.708264089"
Sep 17 15:54:33.271806 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.271786    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jj572" podStartSLOduration=1.624680679 podStartE2EDuration="21.271779037s" totalImagesPullingTime="19.647098358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:33.271471861 +0000 UTC m=+22.708148641" watchObservedRunningTime="2026-09-17 15:54:33.271779037 +0000 UTC m=+22.708455856"
Sep 17 15:54:33.361529 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.361487    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pmbs8" podStartSLOduration=1.737678611 podStartE2EDuration="21.361473487s" totalImagesPullingTime="19.623794876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:33.361320143 +0000 UTC m=+22.797996921" watchObservedRunningTime="2026-09-17 15:54:33.361473487 +0000 UTC m=+22.798150266"
Sep 17 15:54:33.468979 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.468929    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6hslz" podStartSLOduration=1.777138766 podStartE2EDuration="21.468915735s" totalImagesPullingTime="19.691776969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:33.432080076 +0000 UTC m=+22.868756855" watchObservedRunningTime="2026-09-17 15:54:33.468915735 +0000 UTC m=+22.905592513"
Sep 17 15:54:33.469131 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:33.469031    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n7ws2" podStartSLOduration=1.83389439 podStartE2EDuration="21.469024883s" totalImagesPullingTime="19.635130493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:33.463958483 +0000 UTC m=+22.900635261" watchObservedRunningTime="2026-09-17 15:54:33.469024883 +0000 UTC m=+22.905701666"
Sep 17 15:54:34.039781 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:34.039717    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:34.040013 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:34.039832    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:34.040013 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:34.039842    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:34.040013 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:34.039853    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:34.040013 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:34.039945    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:34.040258 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:34.040018    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:34.268495 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:34.268457    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal" event={"ID":"c1ed2aab8eb75b0ce187c7e8a27004a6","Type":"ContainerStarted","Data":"99a75d03150c054a0e8eac8a973b4f45edf396217e75d93f07c3c7958a9c350d"}
Sep 17 15:54:34.270821 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:34.270787    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqrw5" event={"ID":"20531a4e-3ef6-4648-b15e-540ffd02bcc8","Type":"ContainerStarted","Data":"5cc8ff075c835fea6ab5f881d1ffeaa92e108c99b7cde8fc1ec9a5b24678edc9"}
Sep 17 15:54:34.325734 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:34.325640    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-144.ec2.internal" podStartSLOduration=22.32562568 podStartE2EDuration="22.32562568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:34.293783247 +0000 UTC m=+23.730460027" watchObservedRunningTime="2026-09-17 15:54:34.32562568 +0000 UTC m=+23.762302459"
Sep 17 15:54:34.635546 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:34.635514    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 15:54:35.036968 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.036876    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T15:54:34.635540979Z","UUID":"06f4d88b-a605-4b45-be21-7ec65b639952","Handler":null,"Name":"","Endpoint":""}
Sep 17 15:54:35.038532 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.038356    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 15:54:35.038631 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.038548    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 15:54:35.275805 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.275763    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl" event={"ID":"4cd102a9-401a-4aba-974f-361d4f32c6ed","Type":"ContainerStarted","Data":"e99c4234685626de89fdef9ddf845ca3251fefc83d4ae8daedff4e4bf61178fb"}
Sep 17 15:54:35.280198 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.280175    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 15:54:35.280710 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.280685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"ca1bdba74ffd442a0e86ab7c10733655af25381838c19c649477af6e730e6cf5"}
Sep 17 15:54:35.551364 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.551178    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:35.551839 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.551818    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:35.585104 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:35.585052    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lqrw5" podStartSLOduration=3.890940923 podStartE2EDuration="23.585029619s" totalImagesPullingTime="19.694088696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:34.326037518 +0000 UTC m=+23.762714298" watchObservedRunningTime="2026-09-17 15:54:35.585029619 +0000 UTC m=+25.021706403"
Sep 17 15:54:36.039260 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:36.039223    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:36.039468 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:36.039238    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:36.039468 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:36.039346    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:36.039468 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:36.039238    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:36.039468 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:36.039447    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:36.039656 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:36.039489    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:36.282549 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:36.282518    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:36.283075 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:36.283058    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n7ws2"
Sep 17 15:54:38.039226 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.039044    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:38.039569 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.039089    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:38.039569 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:38.039292    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:38.039569 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:38.039367    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:38.039569 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.039101    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:38.039569 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:38.039432    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:38.288957 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.288925    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4244309-a935-44d0-9fe3-3edb7feb081f" containerID="e785bfeaab191ed8e43d5af9e088d6e89ea3133409e56b247b34ebf22144a06c" exitCode=0
Sep 17 15:54:38.289074 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.289002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerDied","Data":"e785bfeaab191ed8e43d5af9e088d6e89ea3133409e56b247b34ebf22144a06c"}
Sep 17 15:54:38.291370 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.291345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl" event={"ID":"4cd102a9-401a-4aba-974f-361d4f32c6ed","Type":"ContainerStarted","Data":"b8495cb20ef4b2e54b863e5fb772f48e28b482459b939df34138ead6d578e02f"}
Sep 17 15:54:38.294570 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.294553    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 15:54:38.294917 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.294899    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"5a6767aea21b2cc83a01faa0dc14e825bfb3974bdafe65f32b101620a8c6f151"}
Sep 17 15:54:38.295301 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:38.295286    2566 scope.go:117] "RemoveContainer" containerID="1c8c5aa893f3c51088bfc32534cae6c1214e76b7644901b581a305705fb7a673"
Sep 17 15:54:39.300868 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.300728    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 15:54:39.301508 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.301193    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" event={"ID":"228d91b2-8567-46ec-a469-c967cb61117c","Type":"ContainerStarted","Data":"8de5828982f5d480c928af111a2196300a622b27a954916574d541672dd200a1"}
Sep 17 15:54:39.301508 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.301451    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:39.301650 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.301549    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:39.301650 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.301594    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:39.303613 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.303585    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4244309-a935-44d0-9fe3-3edb7feb081f" containerID="30698f7444feb29c969c779e1f22c853f64523a96a763c77679b6c380c2b3137" exitCode=0
Sep 17 15:54:39.303729 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.303663    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerDied","Data":"30698f7444feb29c969c779e1f22c853f64523a96a763c77679b6c380c2b3137"}
Sep 17 15:54:39.317295 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.317275    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:39.317383 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.317376    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:54:39.356938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.356894    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s77hl" podStartSLOduration=3.30764911 podStartE2EDuration="27.356881443s" totalImagesPullingTime="24.049232333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:38.397197059 +0000 UTC m=+27.833873837" watchObservedRunningTime="2026-09-17 15:54:39.356881443 +0000 UTC m=+28.793558219"
Sep 17 15:54:39.395248 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.395208    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc" podStartSLOduration=7.48184886 podStartE2EDuration="27.395194478s" totalImagesPullingTime="19.913345618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:39.357563069 +0000 UTC m=+28.794239849" watchObservedRunningTime="2026-09-17 15:54:39.395194478 +0000 UTC m=+28.831871259"
Sep 17 15:54:39.407576 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.407552    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lbkhx"]
Sep 17 15:54:39.407684 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.407647    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:39.407736 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:39.407722    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:39.422430 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.422409    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d4xzc"]
Sep 17 15:54:39.422640 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.422621    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:39.422729 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:39.422701    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:39.423059 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.423038    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vvjn8"]
Sep 17 15:54:39.423136 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:39.423126    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:39.423230 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:39.423214    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:41.039889 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:41.039853    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:41.040355 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:41.039939    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:41.040355 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:41.039954    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:41.040355 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:41.040056    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:41.040355 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:41.040141    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:41.040355 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:41.040052    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:41.310889 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:41.310826    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4244309-a935-44d0-9fe3-3edb7feb081f" containerID="235b74125a2b0c020255d3c3beddcdf184119b9c1ae240e9089b9fc8e2097e6b" exitCode=0
Sep 17 15:54:41.310993 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:41.310902    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerDied","Data":"235b74125a2b0c020255d3c3beddcdf184119b9c1ae240e9089b9fc8e2097e6b"}
Sep 17 15:54:43.040052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:43.040010    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:43.040052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:43.040037    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:43.040585 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:43.040124    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:43.040585 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.040139    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:43.040585 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.040254    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:43.040585 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.040360    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:43.722007 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:43.721976    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:43.722232 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.722133    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:43.722232 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.722217    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:55:15.722196366 +0000 UTC m=+65.158873130 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 15:54:43.822835 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:43.822797    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:43.823003 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.822915    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:43.823003 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.822968    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret podName:9e29d81f-e0e8-43f7-b035-ef4bc9510105 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:15.822956673 +0000 UTC m=+65.259633430 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret") pod "global-pull-secret-syncer-d4xzc" (UID: "9e29d81f-e0e8-43f7-b035-ef4bc9510105") : object "kube-system"/"original-pull-secret" not registered
Sep 17 15:54:43.923957 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:43.923921    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:43.924113 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.924095    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 15:54:43.924191 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.924124    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 15:54:43.924191 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.924140    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5t4qj for pod openshift-network-diagnostics/network-check-target-lbkhx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:43.924312 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:43.924206    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj podName:5152d327-3905-4130-bc08-0e23cf39a6b2 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:15.924192269 +0000 UTC m=+65.360869026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5t4qj" (UniqueName: "kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj") pod "network-check-target-lbkhx" (UID: "5152d327-3905-4130-bc08-0e23cf39a6b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 15:54:45.039547 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:45.039519    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:45.040085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:45.039515    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:45.040085 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:45.039639    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d4xzc" podUID="9e29d81f-e0e8-43f7-b035-ef4bc9510105"
Sep 17 15:54:45.040085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:45.039647    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:45.040085 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:45.039731    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lbkhx" podUID="5152d327-3905-4130-bc08-0e23cf39a6b2"
Sep 17 15:54:45.040085 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:45.039832    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vvjn8" podUID="26f8edd7-4332-49c1-8f85-00a63475c7ed"
Sep 17 15:54:45.903948 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:45.903792    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-144.ec2.internal" event="NodeReady"
Sep 17 15:54:45.904054 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:45.904033    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 15:54:46.017753 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.017679    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6qjsh"]
Sep 17 15:54:46.039968 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.039945    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vrqvr"]
Sep 17 15:54:46.040314 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.040071    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.048192 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.048174    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 15:54:46.049052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.049033    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nhj85\""
Sep 17 15:54:46.049393 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.049377    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 15:54:46.057993 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.057974    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vrqvr"]
Sep 17 15:54:46.058071 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.058018    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6qjsh"]
Sep 17 15:54:46.058071 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.058065    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:54:46.058258 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.058243    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.064338 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.064300    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 15:54:46.064424 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.064355    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 15:54:46.064486 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.064360    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 15:54:46.064711 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.064685    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qr24f\""
Sep 17 15:54:46.064711 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.064696    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 15:54:46.064839 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.064715    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 15:54:46.064839 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.064804    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-djqn9\""
Sep 17 15:54:46.140596 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.140568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.140692 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.140598    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0be3ee88-c394-4754-8569-fde1369da3e1-config-volume\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.140692 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.140621    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be3ee88-c394-4754-8569-fde1369da3e1-tmp-dir\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.140692 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.140659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kx69\" (UniqueName: \"kubernetes.io/projected/9153cbed-c147-4f00-8d05-7015cb79544e-kube-api-access-6kx69\") pod \"ingress-canary-vrqvr\" (UID: \"9153cbed-c147-4f00-8d05-7015cb79544e\") " pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.140792 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.140702    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvx5g\" (UniqueName: \"kubernetes.io/projected/0be3ee88-c394-4754-8569-fde1369da3e1-kube-api-access-rvx5g\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.140792 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.140757    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9153cbed-c147-4f00-8d05-7015cb79544e-cert\") pod \"ingress-canary-vrqvr\" (UID: \"9153cbed-c147-4f00-8d05-7015cb79544e\") " pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.242877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0be3ee88-c394-4754-8569-fde1369da3e1-config-volume\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.242926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be3ee88-c394-4754-8569-fde1369da3e1-tmp-dir\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.242959    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kx69\" (UniqueName: \"kubernetes.io/projected/9153cbed-c147-4f00-8d05-7015cb79544e-kube-api-access-6kx69\") pod \"ingress-canary-vrqvr\" (UID: \"9153cbed-c147-4f00-8d05-7015cb79544e\") " pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.243005    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvx5g\" (UniqueName: \"kubernetes.io/projected/0be3ee88-c394-4754-8569-fde1369da3e1-kube-api-access-rvx5g\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.243086    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9153cbed-c147-4f00-8d05-7015cb79544e-cert\") pod \"ingress-canary-vrqvr\" (UID: \"9153cbed-c147-4f00-8d05-7015cb79544e\") " pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.243129    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:46.243272    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:46.243342    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:46.74331752 +0000 UTC m=+36.179994294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.244097    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0be3ee88-c394-4754-8569-fde1369da3e1-config-volume\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.244559 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.244403    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be3ee88-c394-4754-8569-fde1369da3e1-tmp-dir\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.248794 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.248772    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9153cbed-c147-4f00-8d05-7015cb79544e-cert\") pod \"ingress-canary-vrqvr\" (UID: \"9153cbed-c147-4f00-8d05-7015cb79544e\") " pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.259908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.259880    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kx69\" (UniqueName: \"kubernetes.io/projected/9153cbed-c147-4f00-8d05-7015cb79544e-kube-api-access-6kx69\") pod \"ingress-canary-vrqvr\" (UID: \"9153cbed-c147-4f00-8d05-7015cb79544e\") " pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.261943 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.261920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvx5g\" (UniqueName: \"kubernetes.io/projected/0be3ee88-c394-4754-8569-fde1369da3e1-kube-api-access-rvx5g\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.325096 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.325042    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4244309-a935-44d0-9fe3-3edb7feb081f" containerID="5833359c7ad86e726512cce082260062b01501c0235f7f6d96ce857171f2fc22" exitCode=0
Sep 17 15:54:46.325192 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.325094    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerDied","Data":"5833359c7ad86e726512cce082260062b01501c0235f7f6d96ce857171f2fc22"}
Sep 17 15:54:46.372021 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.371997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vrqvr"
Sep 17 15:54:46.745540 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.745511    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vrqvr"]
Sep 17 15:54:46.746982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:46.746961    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:46.747094 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:46.747081    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:54:46.747144 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:46.747134    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:47.747121238 +0000 UTC m=+37.183797995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:54:46.749496 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:54:46.749473    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9153cbed_c147_4f00_8d05_7015cb79544e.slice/crio-0ac9e9be16e8e0e7229b959454ce6edbb75eacf1ca15cc073dc5e9cf64029c7e WatchSource:0}: Error finding container 0ac9e9be16e8e0e7229b959454ce6edbb75eacf1ca15cc073dc5e9cf64029c7e: Status 404 returned error can't find the container with id 0ac9e9be16e8e0e7229b959454ce6edbb75eacf1ca15cc073dc5e9cf64029c7e
Sep 17 15:54:47.042861 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.042799    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:54:47.043513 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.042949    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:54:47.049273 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.049256    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8v7v5\""
Sep 17 15:54:47.049737 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.049722    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 15:54:47.050290 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.049987    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 15:54:47.331002 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.330928    2566 generic.go:356] "Generic (PLEG): container finished" podID="a4244309-a935-44d0-9fe3-3edb7feb081f" containerID="60c3148a2d26c6d104ee42988e300361309afce76e1db6039082808591882795" exitCode=0
Sep 17 15:54:47.331002 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.330959    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerDied","Data":"60c3148a2d26c6d104ee42988e300361309afce76e1db6039082808591882795"}
Sep 17 15:54:47.332417 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.332392    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vrqvr" event={"ID":"9153cbed-c147-4f00-8d05-7015cb79544e","Type":"ContainerStarted","Data":"0ac9e9be16e8e0e7229b959454ce6edbb75eacf1ca15cc073dc5e9cf64029c7e"}
Sep 17 15:54:47.753654 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:47.753620    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:47.753832 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:47.753788    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:54:47.753911 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:47.753857    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:49.753842001 +0000 UTC m=+39.190518788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:54:48.338956 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:48.338919    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" event={"ID":"a4244309-a935-44d0-9fe3-3edb7feb081f","Type":"ContainerStarted","Data":"6b43902424b6bcb47b592d39cc6ddd903a2d50826335ae413fa10bbcbc793a88"}
Sep 17 15:54:48.373190 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:48.373114    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bwzqr" podStartSLOduration=6.926857009 podStartE2EDuration="36.373094169s" totalImagesPullingTime="29.44623716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:48.371720783 +0000 UTC m=+37.808397593" watchObservedRunningTime="2026-09-17 15:54:48.373094169 +0000 UTC m=+37.809770948"
Sep 17 15:54:49.343122 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:49.343088    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vrqvr" event={"ID":"9153cbed-c147-4f00-8d05-7015cb79544e","Type":"ContainerStarted","Data":"da899d42d8dcd8790ba822de8feab4e6c33e93a77e6cb1a949b1401f2deadfd2"}
Sep 17 15:54:49.363828 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:49.363781    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vrqvr" podStartSLOduration=1.933134644 podStartE2EDuration="4.363767357s" totalImagesPullingTime="2.430632713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:54:49.363628401 +0000 UTC m=+38.800305180" watchObservedRunningTime="2026-09-17 15:54:49.363767357 +0000 UTC m=+38.800444132"
Sep 17 15:54:49.770433 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:49.770400    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:49.770577 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:49.770513    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:54:49.770577 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:49.770572    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:54:53.770559711 +0000 UTC m=+43.207236467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:54:53.795010 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:54:53.794966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:54:53.795416 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:53.795089    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:54:53.795416 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:54:53.795144    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:01.795129937 +0000 UTC m=+51.231806694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:55:01.846994 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:01.846952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:55:01.847462 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:01.847066    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:55:01.847462 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:01.847131    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:17.847116171 +0000 UTC m=+67.283792931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:55:11.321174 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:11.321134    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h25hc"
Sep 17 15:55:15.735629 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.735593    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:55:15.738314 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.738295    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 15:55:15.746673 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:15.746654    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 15:55:15.746734 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:15.746727    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs podName:26f8edd7-4332-49c1-8f85-00a63475c7ed nodeName:}" failed. No retries permitted until 2026-09-17 15:56:19.74670904 +0000 UTC m=+129.183385810 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs") pod "network-metrics-daemon-vvjn8" (UID: "26f8edd7-4332-49c1-8f85-00a63475c7ed") : secret "metrics-daemon-secret" not found
Sep 17 15:55:15.836499 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.836473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:55:15.841320 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.841302    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 15:55:15.850547 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.850522    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e29d81f-e0e8-43f7-b035-ef4bc9510105-original-pull-secret\") pod \"global-pull-secret-syncer-d4xzc\" (UID: \"9e29d81f-e0e8-43f7-b035-ef4bc9510105\") " pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:55:15.852459 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.852442    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d4xzc"
Sep 17 15:55:15.937621 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.937574    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:55:15.943516 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.943491    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 15:55:15.953553 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.953521    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 15:55:15.961096 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:15.961074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4qj\" (UniqueName: \"kubernetes.io/projected/5152d327-3905-4130-bc08-0e23cf39a6b2-kube-api-access-5t4qj\") pod \"network-check-target-lbkhx\" (UID: \"5152d327-3905-4130-bc08-0e23cf39a6b2\") " pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:55:16.071189 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:16.071120    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qr24f\""
Sep 17 15:55:16.078164 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:16.078138    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:55:16.159730 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:16.159704    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d4xzc"]
Sep 17 15:55:16.163065 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:55:16.163036    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e29d81f_e0e8_43f7_b035_ef4bc9510105.slice/crio-9c96ca98af2a62a1b28ef0c464427b1d4e10a78572fff13f35f7c0c3a22b8016 WatchSource:0}: Error finding container 9c96ca98af2a62a1b28ef0c464427b1d4e10a78572fff13f35f7c0c3a22b8016: Status 404 returned error can't find the container with id 9c96ca98af2a62a1b28ef0c464427b1d4e10a78572fff13f35f7c0c3a22b8016
Sep 17 15:55:16.369052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:16.368973    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lbkhx"]
Sep 17 15:55:16.372131 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:55:16.372108    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5152d327_3905_4130_bc08_0e23cf39a6b2.slice/crio-94f92553bbce2955d4eb5bcf92d3d902a7b24783833fe47d5c807f5664a6d485 WatchSource:0}: Error finding container 94f92553bbce2955d4eb5bcf92d3d902a7b24783833fe47d5c807f5664a6d485: Status 404 returned error can't find the container with id 94f92553bbce2955d4eb5bcf92d3d902a7b24783833fe47d5c807f5664a6d485
Sep 17 15:55:16.403644 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:16.403613    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d4xzc" event={"ID":"9e29d81f-e0e8-43f7-b035-ef4bc9510105","Type":"ContainerStarted","Data":"9c96ca98af2a62a1b28ef0c464427b1d4e10a78572fff13f35f7c0c3a22b8016"}
Sep 17 15:55:16.404716 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:16.404690    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lbkhx" event={"ID":"5152d327-3905-4130-bc08-0e23cf39a6b2","Type":"ContainerStarted","Data":"94f92553bbce2955d4eb5bcf92d3d902a7b24783833fe47d5c807f5664a6d485"}
Sep 17 15:55:17.847778 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:17.847735    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:55:17.848357 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:17.847880    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:55:17.848357 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:17.847947    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:49.847928946 +0000 UTC m=+99.284605709 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:55:21.422361 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:21.422276    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lbkhx" event={"ID":"5152d327-3905-4130-bc08-0e23cf39a6b2","Type":"ContainerStarted","Data":"7633e60ebfb25698f4756ac027f4bde5e68decbded889253b59cd5ade43c95ed"}
Sep 17 15:55:21.422726 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:21.422405    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:55:21.423865 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:21.423842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d4xzc" event={"ID":"9e29d81f-e0e8-43f7-b035-ef4bc9510105","Type":"ContainerStarted","Data":"6745c58ef75a7ad4613194a59c6ce75e2b6d2d8232823b012f9ed8e72c3b1be7"}
Sep 17 15:55:21.445534 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:21.445485    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lbkhx" podStartSLOduration=64.76732414 podStartE2EDuration="1m9.445472676s" totalImagesPullingTime="4.678148536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:55:21.444380428 +0000 UTC m=+70.881057207" watchObservedRunningTime="2026-09-17 15:55:21.445472676 +0000 UTC m=+70.882149455"
Sep 17 15:55:21.470966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:21.470924    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d4xzc" podStartSLOduration=64.578355596 podStartE2EDuration="1m9.470886586s" totalImagesPullingTime="4.89253099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:55:21.470139731 +0000 UTC m=+70.906816510" watchObservedRunningTime="2026-09-17 15:55:21.470886586 +0000 UTC m=+70.907563364"
Sep 17 15:55:49.941339 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:49.941283    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:55:49.941779 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:49.941431    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 15:55:49.941779 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:49.941517    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls podName:0be3ee88-c394-4754-8569-fde1369da3e1 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.941500353 +0000 UTC m=+163.378177114 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls") pod "dns-default-6qjsh" (UID: "0be3ee88-c394-4754-8569-fde1369da3e1") : secret "dns-default-metrics-tls" not found
Sep 17 15:55:52.429327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:52.429297    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lbkhx"
Sep 17 15:55:55.845127 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.845091    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-btkfl"]
Sep 17 15:55:55.875513 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.875475    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-btkfl"]
Sep 17 15:55:55.875656 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.875508    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:55.878712 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.878686    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 17 15:55:55.878858 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.878689    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 17 15:55:55.879107 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.879088    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4hzxv\""
Sep 17 15:55:55.880272 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.880253    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 15:55:55.880473 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.880257    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 15:55:55.885349 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.885329    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 17 15:55:55.981365 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.981330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ccb002d7-89bf-4dd2-8909-8701ab7755f6-snapshots\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:55.981580 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.981391    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb002d7-89bf-4dd2-8909-8701ab7755f6-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:55.981580 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.981430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfd4g\" (UniqueName: \"kubernetes.io/projected/ccb002d7-89bf-4dd2-8909-8701ab7755f6-kube-api-access-lfd4g\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:55.981580 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.981451    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccb002d7-89bf-4dd2-8909-8701ab7755f6-tmp\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:55.981580 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.981519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ccb002d7-89bf-4dd2-8909-8701ab7755f6-serving-cert\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:55.981580 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:55.981551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb002d7-89bf-4dd2-8909-8701ab7755f6-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.015809 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.015777    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-2zp6f"]
Sep 17 15:55:56.036768 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.036742    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"]
Sep 17 15:55:56.036901 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.036881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.050259 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.050238    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 17 15:55:56.051923 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.051900    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 17 15:55:56.052143 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.052123    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 17 15:55:56.052263 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.052250    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9dgxj\""
Sep 17 15:55:56.053047 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.053020    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.053838 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.053816    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-2zp6f"]
Sep 17 15:55:56.054483 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.054464    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"]
Sep 17 15:55:56.055698 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.055677    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:55:56.056238 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.056218    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 15:55:56.056704 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.056683    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l6qpv\""
Sep 17 15:55:56.056904 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.056882    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 17 15:55:56.057026 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.057010    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 15:55:56.059794 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.059776    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 17 15:55:56.063807 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.063790    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 17 15:55:56.082744 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082718    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfd4g\" (UniqueName: \"kubernetes.io/projected/ccb002d7-89bf-4dd2-8909-8701ab7755f6-kube-api-access-lfd4g\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.082836 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082756    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccb002d7-89bf-4dd2-8909-8701ab7755f6-tmp\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.082836 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082779    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ccb002d7-89bf-4dd2-8909-8701ab7755f6-serving-cert\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.082836 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kctd7\" (UniqueName: \"kubernetes.io/projected/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-kube-api-access-kctd7\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.082986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082838    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c520bd2-3078-4ef1-a685-70e71b9467af-trusted-ca\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.082986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb002d7-89bf-4dd2-8909-8701ab7755f6-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.082986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082898    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c520bd2-3078-4ef1-a685-70e71b9467af-serving-cert\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.082986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ccb002d7-89bf-4dd2-8909-8701ab7755f6-snapshots\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.082986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.082955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsvd5\" (UniqueName: \"kubernetes.io/projected/6c520bd2-3078-4ef1-a685-70e71b9467af-kube-api-access-tsvd5\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.083261 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.083261 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083190    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c520bd2-3078-4ef1-a685-70e71b9467af-config\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.083261 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083209    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.083261 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083188    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccb002d7-89bf-4dd2-8909-8701ab7755f6-tmp\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.083441 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083349    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb002d7-89bf-4dd2-8909-8701ab7755f6-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.083699 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ccb002d7-89bf-4dd2-8909-8701ab7755f6-snapshots\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.083895 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083876    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb002d7-89bf-4dd2-8909-8701ab7755f6-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.083983 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.083965    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb002d7-89bf-4dd2-8909-8701ab7755f6-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.085468 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.085440    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ccb002d7-89bf-4dd2-8909-8701ab7755f6-serving-cert\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.099369 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.099317    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"]
Sep 17 15:55:56.110797 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.110776    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfd4g\" (UniqueName: \"kubernetes.io/projected/ccb002d7-89bf-4dd2-8909-8701ab7755f6-kube-api-access-lfd4g\") pod \"insights-operator-6f9c644dd9-btkfl\" (UID: \"ccb002d7-89bf-4dd2-8909-8701ab7755f6\") " pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.121730 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.121714    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.124603 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.124580    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 17 15:55:56.124932 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.124915    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"]
Sep 17 15:55:56.125525 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.125503    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 17 15:55:56.125525 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.125517    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 17 15:55:56.125676 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.125575    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 17 15:55:56.125748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.125728    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ffr94\""
Sep 17 15:55:56.184044 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184017    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kctd7\" (UniqueName: \"kubernetes.io/projected/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-kube-api-access-kctd7\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.184044 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184047    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c520bd2-3078-4ef1-a685-70e71b9467af-trusted-ca\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.184296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f02766b9-9dda-442e-8486-b9865cae794f-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.184296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184088    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c520bd2-3078-4ef1-a685-70e71b9467af-serving-cert\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.184296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsvd5\" (UniqueName: \"kubernetes.io/projected/6c520bd2-3078-4ef1-a685-70e71b9467af-kube-api-access-tsvd5\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.184296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184136    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.184296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c520bd2-3078-4ef1-a685-70e71b9467af-config\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.184296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184199    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.184296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9zpv\" (UniqueName: \"kubernetes.io/projected/f02766b9-9dda-442e-8486-b9865cae794f-kube-api-access-l9zpv\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.184631 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f02766b9-9dda-442e-8486-b9865cae794f-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.184631 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.184339    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:56.184631 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184433    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-btkfl"
Sep 17 15:55:56.184631 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.184459    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls podName:8e0f92ca-79a8-429d-bdd5-e2924cc47b6e nodeName:}" failed. No retries permitted until 2026-09-17 15:55:56.684437416 +0000 UTC m=+106.121114186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-rs89x" (UID: "8e0f92ca-79a8-429d-bdd5-e2924cc47b6e") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:56.184897 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184880    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.184938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c520bd2-3078-4ef1-a685-70e71b9467af-config\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.184971 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.184948    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6c520bd2-3078-4ef1-a685-70e71b9467af-trusted-ca\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.186532 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.186515    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c520bd2-3078-4ef1-a685-70e71b9467af-serving-cert\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.192964 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.192940    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66cf577dcf-2wh9p"]
Sep 17 15:55:56.201654 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.201635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kctd7\" (UniqueName: \"kubernetes.io/projected/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-kube-api-access-kctd7\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.202088 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.202068    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsvd5\" (UniqueName: \"kubernetes.io/projected/6c520bd2-3078-4ef1-a685-70e71b9467af-kube-api-access-tsvd5\") pod \"console-operator-fd4bb46c-2zp6f\" (UID: \"6c520bd2-3078-4ef1-a685-70e71b9467af\") " pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.222019 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.221991    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66cf577dcf-2wh9p"]
Sep 17 15:55:56.222125 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.222081    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.224997 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.224976    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 15:55:56.225613 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.225412    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 15:55:56.225613 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.225480    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 15:55:56.225613 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.225504    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7jqdh\""
Sep 17 15:55:56.232107 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.232088    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 15:55:56.285357 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285271    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-installation-pull-secrets\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285357 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-image-registry-private-configuration\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285357 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285349    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285557 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9zpv\" (UniqueName: \"kubernetes.io/projected/f02766b9-9dda-442e-8486-b9865cae794f-kube-api-access-l9zpv\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.285557 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285407    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f02766b9-9dda-442e-8486-b9865cae794f-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.285557 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285424    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-trusted-ca\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285557 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285447    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-bound-sa-token\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285557 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b24q\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-kube-api-access-4b24q\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285589    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f02766b9-9dda-442e-8486-b9865cae794f-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.285703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285627    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2ab36b25-1767-452f-8b74-520b39589954-ca-trust-extracted\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285654    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-registry-certificates\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.285950 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.285933    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f02766b9-9dda-442e-8486-b9865cae794f-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.287615 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.287594    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f02766b9-9dda-442e-8486-b9865cae794f-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.295689 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.295666    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9zpv\" (UniqueName: \"kubernetes.io/projected/f02766b9-9dda-442e-8486-b9865cae794f-kube-api-access-l9zpv\") pod \"kube-storage-version-migrator-operator-55889f7b47-7564n\" (UID: \"f02766b9-9dda-442e-8486-b9865cae794f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.347604 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.347574    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:55:56.386913 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.386885    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-installation-pull-secrets\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387022 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.386930    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-image-registry-private-configuration\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387022 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.386950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387022 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.386986    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-trusted-ca\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387022 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.387001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-bound-sa-token\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387022 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.387019    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b24q\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-kube-api-access-4b24q\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387504 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.387072    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2ab36b25-1767-452f-8b74-520b39589954-ca-trust-extracted\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387504 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.387096    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-registry-certificates\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387504 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.387141    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:55:56.387504 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.387174    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66cf577dcf-2wh9p: secret "image-registry-tls" not found
Sep 17 15:55:56.387504 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.387236    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls podName:2ab36b25-1767-452f-8b74-520b39589954 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:56.887215007 +0000 UTC m=+106.323891782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls") pod "image-registry-66cf577dcf-2wh9p" (UID: "2ab36b25-1767-452f-8b74-520b39589954") : secret "image-registry-tls" not found
Sep 17 15:55:56.387755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.387655    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2ab36b25-1767-452f-8b74-520b39589954-ca-trust-extracted\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.387755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.387700    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-registry-certificates\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.388083 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.388062    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-trusted-ca\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.389143 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.389124    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-installation-pull-secrets\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.389267 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.389219    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-image-registry-private-configuration\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.397124 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.397102    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b24q\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-kube-api-access-4b24q\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.397225 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.397151    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-bound-sa-token\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.430493 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.430465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"
Sep 17 15:55:56.488770 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.488742    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-btkfl"]
Sep 17 15:55:56.491391 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:55:56.491368    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb002d7_89bf_4dd2_8909_8701ab7755f6.slice/crio-eee31cd39ad5473f61972e12f6bf135f584b6325abe76b42c04578156e772d9d WatchSource:0}: Error finding container eee31cd39ad5473f61972e12f6bf135f584b6325abe76b42c04578156e772d9d: Status 404 returned error can't find the container with id eee31cd39ad5473f61972e12f6bf135f584b6325abe76b42c04578156e772d9d
Sep 17 15:55:56.506917 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.506891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-btkfl" event={"ID":"ccb002d7-89bf-4dd2-8909-8701ab7755f6","Type":"ContainerStarted","Data":"eee31cd39ad5473f61972e12f6bf135f584b6325abe76b42c04578156e772d9d"}
Sep 17 15:55:56.653151 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.651599    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-2zp6f"]
Sep 17 15:55:56.657893 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:55:56.657862    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c520bd2_3078_4ef1_a685_70e71b9467af.slice/crio-40e4fbf7ff1389d6e407a29b44e0e947f8c5778d0e7bbfdc40aaab7fcf65ff1c WatchSource:0}: Error finding container 40e4fbf7ff1389d6e407a29b44e0e947f8c5778d0e7bbfdc40aaab7fcf65ff1c: Status 404 returned error can't find the container with id 40e4fbf7ff1389d6e407a29b44e0e947f8c5778d0e7bbfdc40aaab7fcf65ff1c
Sep 17 15:55:56.688631 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.688604    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:56.688744 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.688729    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:56.688785 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.688779    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls podName:8e0f92ca-79a8-429d-bdd5-e2924cc47b6e nodeName:}" failed. No retries permitted until 2026-09-17 15:55:57.688766411 +0000 UTC m=+107.125443167 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-rs89x" (UID: "8e0f92ca-79a8-429d-bdd5-e2924cc47b6e") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:56.717520 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.717497    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n"]
Sep 17 15:55:56.718526 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:55:56.718502    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02766b9_9dda_442e_8486_b9865cae794f.slice/crio-b7c9e2105f9d1c7ea3ff0431ae55a7a5b349e91b7d6a787cb966f06cf2d7f890 WatchSource:0}: Error finding container b7c9e2105f9d1c7ea3ff0431ae55a7a5b349e91b7d6a787cb966f06cf2d7f890: Status 404 returned error can't find the container with id b7c9e2105f9d1c7ea3ff0431ae55a7a5b349e91b7d6a787cb966f06cf2d7f890
Sep 17 15:55:56.889984 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:56.889957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:56.890386 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.890134    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:55:56.890386 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.890172    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66cf577dcf-2wh9p: secret "image-registry-tls" not found
Sep 17 15:55:56.890386 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:56.890236    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls podName:2ab36b25-1767-452f-8b74-520b39589954 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:57.89021583 +0000 UTC m=+107.326892607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls") pod "image-registry-66cf577dcf-2wh9p" (UID: "2ab36b25-1767-452f-8b74-520b39589954") : secret "image-registry-tls" not found
Sep 17 15:55:57.511905 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:57.511864    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" event={"ID":"6c520bd2-3078-4ef1-a685-70e71b9467af","Type":"ContainerStarted","Data":"40e4fbf7ff1389d6e407a29b44e0e947f8c5778d0e7bbfdc40aaab7fcf65ff1c"}
Sep 17 15:55:57.513503 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:57.513469    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n" event={"ID":"f02766b9-9dda-442e-8486-b9865cae794f","Type":"ContainerStarted","Data":"b7c9e2105f9d1c7ea3ff0431ae55a7a5b349e91b7d6a787cb966f06cf2d7f890"}
Sep 17 15:55:57.696492 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:57.696420    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:57.696647 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:57.696595    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:57.696708 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:57.696664    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls podName:8e0f92ca-79a8-429d-bdd5-e2924cc47b6e nodeName:}" failed. No retries permitted until 2026-09-17 15:55:59.696645273 +0000 UTC m=+109.133322047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-rs89x" (UID: "8e0f92ca-79a8-429d-bdd5-e2924cc47b6e") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:57.897948 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:57.897761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:57.897948 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:57.897940    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:55:57.898467 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:57.897959    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66cf577dcf-2wh9p: secret "image-registry-tls" not found
Sep 17 15:55:57.898467 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:57.898020    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls podName:2ab36b25-1767-452f-8b74-520b39589954 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:59.898002618 +0000 UTC m=+109.334679392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls") pod "image-registry-66cf577dcf-2wh9p" (UID: "2ab36b25-1767-452f-8b74-520b39589954") : secret "image-registry-tls" not found
Sep 17 15:55:59.521361 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.521287    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-btkfl" event={"ID":"ccb002d7-89bf-4dd2-8909-8701ab7755f6","Type":"ContainerStarted","Data":"3b1db79201353dc8b49a2c7b71482d2893a819dc395322acaeb10813becdad41"}
Sep 17 15:55:59.523006 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.522984    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/0.log"
Sep 17 15:55:59.523086 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.523027    2566 generic.go:356] "Generic (PLEG): container finished" podID="6c520bd2-3078-4ef1-a685-70e71b9467af" containerID="37e4b0bc49baf3063a8cd118b6e9e5409d0adfdb25dcc691092f310a2ac1d855" exitCode=255
Sep 17 15:55:59.523147 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.523086    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" event={"ID":"6c520bd2-3078-4ef1-a685-70e71b9467af","Type":"ContainerDied","Data":"37e4b0bc49baf3063a8cd118b6e9e5409d0adfdb25dcc691092f310a2ac1d855"}
Sep 17 15:55:59.523300 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.523286    2566 scope.go:117] "RemoveContainer" containerID="37e4b0bc49baf3063a8cd118b6e9e5409d0adfdb25dcc691092f310a2ac1d855"
Sep 17 15:55:59.524750 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.524726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n" event={"ID":"f02766b9-9dda-442e-8486-b9865cae794f","Type":"ContainerStarted","Data":"156990274c9a020b36258153015208b99b73f7dc4124a787f58007e9369bcbe0"}
Sep 17 15:55:59.540427 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.540208    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-btkfl" podStartSLOduration=1.7573576869999998 podStartE2EDuration="4.540190035s" totalImagesPullingTime="2.782832348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:55:59.540067851 +0000 UTC m=+108.976744632" watchObservedRunningTime="2026-09-17 15:55:59.540190035 +0000 UTC m=+108.976866814"
Sep 17 15:55:59.575022 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.574972    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n" podStartSLOduration=1.016087884 podStartE2EDuration="3.574954103s" totalImagesPullingTime="2.558866219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:55:59.574720887 +0000 UTC m=+109.011397666" watchObservedRunningTime="2026-09-17 15:55:59.574954103 +0000 UTC m=+109.011630883"
Sep 17 15:55:59.713568 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.713538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:55:59.713698 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:59.713667    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:59.713774 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:59.713722    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls podName:8e0f92ca-79a8-429d-bdd5-e2924cc47b6e nodeName:}" failed. No retries permitted until 2026-09-17 15:56:03.713709821 +0000 UTC m=+113.150386592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-rs89x" (UID: "8e0f92ca-79a8-429d-bdd5-e2924cc47b6e") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:55:59.914863 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:55:59.914786    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:55:59.915013 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:59.914934    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:55:59.915013 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:59.914951    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66cf577dcf-2wh9p: secret "image-registry-tls" not found
Sep 17 15:55:59.915013 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:55:59.915006    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls podName:2ab36b25-1767-452f-8b74-520b39589954 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:03.914989083 +0000 UTC m=+113.351665844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls") pod "image-registry-66cf577dcf-2wh9p" (UID: "2ab36b25-1767-452f-8b74-520b39589954") : secret "image-registry-tls" not found
Sep 17 15:56:00.334872 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.334842    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ql9hb"]
Sep 17 15:56:00.337979 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.337963    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.340447 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.340419    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hkffj\""
Sep 17 15:56:00.340568 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.340416    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 15:56:00.341281 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.341267    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 15:56:00.365171 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.365125    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ql9hb"]
Sep 17 15:56:00.418522 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.418492    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9162039-1eba-40eb-af07-47697bc0cc05-data-volume\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.418660 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.418527    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ccc8\" (UniqueName: \"kubernetes.io/projected/d9162039-1eba-40eb-af07-47697bc0cc05-kube-api-access-6ccc8\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.418660 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.418553    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.418660 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.418613    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.418759 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.418706    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9162039-1eba-40eb-af07-47697bc0cc05-crio-socket\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519200 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.519171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9162039-1eba-40eb-af07-47697bc0cc05-data-volume\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519360 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.519207    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccc8\" (UniqueName: \"kubernetes.io/projected/d9162039-1eba-40eb-af07-47697bc0cc05-kube-api-access-6ccc8\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519360 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.519231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519360 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:00.519294    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:56:00.519360 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:00.519346    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:01.019333071 +0000 UTC m=+110.456009828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : configmap "kube-rbac-proxy" not found
Sep 17 15:56:00.519360 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.519359    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519609 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.519414    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9162039-1eba-40eb-af07-47697bc0cc05-crio-socket\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519609 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.519476    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9162039-1eba-40eb-af07-47697bc0cc05-crio-socket\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519609 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.519511    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9162039-1eba-40eb-af07-47697bc0cc05-data-volume\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.519609 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:00.519527    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:00.519609 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:00.519575    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:01.019563179 +0000 UTC m=+110.456239941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:00.529071 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.529045    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ccc8\" (UniqueName: \"kubernetes.io/projected/d9162039-1eba-40eb-af07-47697bc0cc05-kube-api-access-6ccc8\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:00.530087 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.530071    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/1.log"
Sep 17 15:56:00.530566 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.530546    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/0.log"
Sep 17 15:56:00.530625 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.530591    2566 generic.go:356] "Generic (PLEG): container finished" podID="6c520bd2-3078-4ef1-a685-70e71b9467af" containerID="c47444815e7aab3e4ffe3d82bfa971f599a5ed7607b9536cab09cf7fe94f04bb" exitCode=255
Sep 17 15:56:00.530687 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.530669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" event={"ID":"6c520bd2-3078-4ef1-a685-70e71b9467af","Type":"ContainerDied","Data":"c47444815e7aab3e4ffe3d82bfa971f599a5ed7607b9536cab09cf7fe94f04bb"}
Sep 17 15:56:00.530726 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.530720    2566 scope.go:117] "RemoveContainer" containerID="37e4b0bc49baf3063a8cd118b6e9e5409d0adfdb25dcc691092f310a2ac1d855"
Sep 17 15:56:00.530956 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:00.530935    2566 scope.go:117] "RemoveContainer" containerID="c47444815e7aab3e4ffe3d82bfa971f599a5ed7607b9536cab09cf7fe94f04bb"
Sep 17 15:56:00.531134 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:00.531108    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-2zp6f_openshift-console-operator(6c520bd2-3078-4ef1-a685-70e71b9467af)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" podUID="6c520bd2-3078-4ef1-a685-70e71b9467af"
Sep 17 15:56:01.023970 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:01.023937    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:01.023970 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:01.023974    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:01.024283 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:01.024060    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:56:01.024283 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:01.024097    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:01.024283 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:01.024137    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:02.02412017 +0000 UTC m=+111.460796948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : configmap "kube-rbac-proxy" not found
Sep 17 15:56:01.024283 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:01.024151    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:02.024145436 +0000 UTC m=+111.460822193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:01.535672 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:01.535646    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/1.log"
Sep 17 15:56:01.536034 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:01.535982    2566 scope.go:117] "RemoveContainer" containerID="c47444815e7aab3e4ffe3d82bfa971f599a5ed7607b9536cab09cf7fe94f04bb"
Sep 17 15:56:01.536182 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:01.536145    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-2zp6f_openshift-console-operator(6c520bd2-3078-4ef1-a685-70e71b9467af)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" podUID="6c520bd2-3078-4ef1-a685-70e71b9467af"
Sep 17 15:56:02.032105 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.032071    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:02.032294 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.032112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:02.032294 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:02.032216    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:56:02.032294 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:02.032233    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:02.032294 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:02.032280    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:04.032268108 +0000 UTC m=+113.468944878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:02.032294 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:02.032296    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:04.032289419 +0000 UTC m=+113.468966188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : configmap "kube-rbac-proxy" not found
Sep 17 15:56:02.260302 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.260271    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pmbs8_71cbb99d-076e-430c-92a1-a135f7c133b8/dns-node-resolver/0.log"
Sep 17 15:56:02.871383 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.871350    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"]
Sep 17 15:56:02.874383 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.874365    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"
Sep 17 15:56:02.877040 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.877020    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wpscb\""
Sep 17 15:56:02.887314 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.887294    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"]
Sep 17 15:56:02.936849 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:02.936821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbjpv\" (UniqueName: \"kubernetes.io/projected/bac97315-91c7-4dee-9c2a-3b05a65454e0-kube-api-access-lbjpv\") pod \"network-check-source-7cbc45fddc-zfb4v\" (UID: \"bac97315-91c7-4dee-9c2a-3b05a65454e0\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"
Sep 17 15:56:03.038228 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.038182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbjpv\" (UniqueName: \"kubernetes.io/projected/bac97315-91c7-4dee-9c2a-3b05a65454e0-kube-api-access-lbjpv\") pod \"network-check-source-7cbc45fddc-zfb4v\" (UID: \"bac97315-91c7-4dee-9c2a-3b05a65454e0\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"
Sep 17 15:56:03.050944 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.050924    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbjpv\" (UniqueName: \"kubernetes.io/projected/bac97315-91c7-4dee-9c2a-3b05a65454e0-kube-api-access-lbjpv\") pod \"network-check-source-7cbc45fddc-zfb4v\" (UID: \"bac97315-91c7-4dee-9c2a-3b05a65454e0\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"
Sep 17 15:56:03.182853 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.182781    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"
Sep 17 15:56:03.456564 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.456482    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v"]
Sep 17 15:56:03.456727 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.456581    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jj572_4c789c20-2a1f-4b93-8b12-d9c8144c53a7/node-ca/0.log"
Sep 17 15:56:03.459217 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:03.459192    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbac97315_91c7_4dee_9c2a_3b05a65454e0.slice/crio-f92ed0523cb3065aef83664df7f02825cdcd68d8d7e7d03332c6b08dc657790c WatchSource:0}: Error finding container f92ed0523cb3065aef83664df7f02825cdcd68d8d7e7d03332c6b08dc657790c: Status 404 returned error can't find the container with id f92ed0523cb3065aef83664df7f02825cdcd68d8d7e7d03332c6b08dc657790c
Sep 17 15:56:03.544106 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.544072    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v" event={"ID":"bac97315-91c7-4dee-9c2a-3b05a65454e0","Type":"ContainerStarted","Data":"ecc7da0916f0098f845941953979cb286e59606a0ed14fa8e79107e75467eefc"}
Sep 17 15:56:03.544337 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.544317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v" event={"ID":"bac97315-91c7-4dee-9c2a-3b05a65454e0","Type":"ContainerStarted","Data":"f92ed0523cb3065aef83664df7f02825cdcd68d8d7e7d03332c6b08dc657790c"}
Sep 17 15:56:03.595238 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.595189    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-zfb4v" podStartSLOduration=1.595173986 podStartE2EDuration="1.595173986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:03.587445376 +0000 UTC m=+113.024122155" watchObservedRunningTime="2026-09-17 15:56:03.595173986 +0000 UTC m=+113.031850756"
Sep 17 15:56:03.743719 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.743691    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:56:03.743835 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:03.743789    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 15:56:03.743879 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:03.743860    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls podName:8e0f92ca-79a8-429d-bdd5-e2924cc47b6e nodeName:}" failed. No retries permitted until 2026-09-17 15:56:11.743847061 +0000 UTC m=+121.180523817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-rs89x" (UID: "8e0f92ca-79a8-429d-bdd5-e2924cc47b6e") : secret "cluster-monitoring-operator-tls" not found
Sep 17 15:56:03.945693 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:03.945665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:56:03.946040 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:03.945767    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 15:56:03.946040 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:03.945779    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66cf577dcf-2wh9p: secret "image-registry-tls" not found
Sep 17 15:56:03.946040 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:03.945820    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls podName:2ab36b25-1767-452f-8b74-520b39589954 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:11.945808933 +0000 UTC m=+121.382485691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls") pod "image-registry-66cf577dcf-2wh9p" (UID: "2ab36b25-1767-452f-8b74-520b39589954") : secret "image-registry-tls" not found
Sep 17 15:56:04.046445 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:04.046381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:04.046445 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:04.046422    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:04.046616 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:04.046492    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:56:04.046616 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:04.046527    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:04.046616 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:04.046548    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:08.04653298 +0000 UTC m=+117.483209763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : configmap "kube-rbac-proxy" not found
Sep 17 15:56:04.046616 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:04.046573    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:08.0465587 +0000 UTC m=+117.483235473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:04.268452 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:04.268418    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vrqvr_9153cbed-c147-4f00-8d05-7015cb79544e/serve-healthcheck-canary/0.log"
Sep 17 15:56:06.347685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:06.347652    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:56:06.348052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:06.347713    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:56:06.348052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:06.348020    2566 scope.go:117] "RemoveContainer" containerID="c47444815e7aab3e4ffe3d82bfa971f599a5ed7607b9536cab09cf7fe94f04bb"
Sep 17 15:56:06.348207 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:06.348184    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-2zp6f_openshift-console-operator(6c520bd2-3078-4ef1-a685-70e71b9467af)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" podUID="6c520bd2-3078-4ef1-a685-70e71b9467af"
Sep 17 15:56:08.076515 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:08.076481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:08.076515 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:08.076518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:08.076932 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:08.076594    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 15:56:08.076932 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:08.076654    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:16.076640715 +0000 UTC m=+125.513317472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : configmap "kube-rbac-proxy" not found
Sep 17 15:56:08.076932 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:08.076665    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:08.076932 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:08.076709    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls podName:d9162039-1eba-40eb-af07-47697bc0cc05 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:16.076693305 +0000 UTC m=+125.513370080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ql9hb" (UID: "d9162039-1eba-40eb-af07-47697bc0cc05") : secret "insights-runtime-extractor-tls" not found
Sep 17 15:56:11.804404 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:11.804366    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:56:11.806735 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:11.806711    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0f92ca-79a8-429d-bdd5-e2924cc47b6e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-rs89x\" (UID: \"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:56:11.965990 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:11.965961    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l6qpv\""
Sep 17 15:56:11.973289 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:11.973266    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"
Sep 17 15:56:12.006287 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.006259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:56:12.008579 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.008537    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"image-registry-66cf577dcf-2wh9p\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") " pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:56:12.137915 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.137842    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7jqdh\""
Sep 17 15:56:12.142307 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.142289    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:56:12.268293 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.268266    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x"]
Sep 17 15:56:12.271300 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:12.271277    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e0f92ca_79a8_429d_bdd5_e2924cc47b6e.slice/crio-01ee874c1a3cb81753ff13dd61850967278c28b86b360d25c9d68911bf25d444 WatchSource:0}: Error finding container 01ee874c1a3cb81753ff13dd61850967278c28b86b360d25c9d68911bf25d444: Status 404 returned error can't find the container with id 01ee874c1a3cb81753ff13dd61850967278c28b86b360d25c9d68911bf25d444
Sep 17 15:56:12.419914 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.419851    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66cf577dcf-2wh9p"]
Sep 17 15:56:12.422523 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:12.422500    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab36b25_1767_452f_8b74_520b39589954.slice/crio-ea5687f02da5c10f515891b71d7b2f35296b0119f29f692e8d955ca37819842c WatchSource:0}: Error finding container ea5687f02da5c10f515891b71d7b2f35296b0119f29f692e8d955ca37819842c: Status 404 returned error can't find the container with id ea5687f02da5c10f515891b71d7b2f35296b0119f29f692e8d955ca37819842c
Sep 17 15:56:12.573796 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.573747    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" event={"ID":"2ab36b25-1767-452f-8b74-520b39589954","Type":"ContainerStarted","Data":"21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd"}
Sep 17 15:56:12.573981 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.573853    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" event={"ID":"2ab36b25-1767-452f-8b74-520b39589954","Type":"ContainerStarted","Data":"ea5687f02da5c10f515891b71d7b2f35296b0119f29f692e8d955ca37819842c"}
Sep 17 15:56:12.573981 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.573890    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:56:12.575037 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.575013    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x" event={"ID":"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e","Type":"ContainerStarted","Data":"01ee874c1a3cb81753ff13dd61850967278c28b86b360d25c9d68911bf25d444"}
Sep 17 15:56:12.604144 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:12.604094    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" podStartSLOduration=16.604078689 podStartE2EDuration="16.604078689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:12.603977214 +0000 UTC m=+122.040653992" watchObservedRunningTime="2026-09-17 15:56:12.604078689 +0000 UTC m=+122.040755467"
Sep 17 15:56:14.582437 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:14.582401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x" event={"ID":"8e0f92ca-79a8-429d-bdd5-e2924cc47b6e","Type":"ContainerStarted","Data":"a8c75cead7feaaf0ac4125fb67e9dc3c366a23f1e89ddf0515623c822f839032"}
Sep 17 15:56:14.609754 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:14.609705    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-rs89x" podStartSLOduration=18.004742061 podStartE2EDuration="19.609690651s" totalImagesPullingTime="1.60494859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:14.60919249 +0000 UTC m=+124.045869263" watchObservedRunningTime="2026-09-17 15:56:14.609690651 +0000 UTC m=+124.046367430"
Sep 17 15:56:16.144010 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.143973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:16.144010 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.144013    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:16.144511 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.144491    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9162039-1eba-40eb-af07-47697bc0cc05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:16.146337 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.146314    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9162039-1eba-40eb-af07-47697bc0cc05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ql9hb\" (UID: \"d9162039-1eba-40eb-af07-47697bc0cc05\") " pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:16.250270 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.250244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hkffj\""
Sep 17 15:56:16.258667 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.258648    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ql9hb"
Sep 17 15:56:16.538280 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.538248    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ql9hb"]
Sep 17 15:56:16.541123 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:16.541091    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9162039_1eba_40eb_af07_47697bc0cc05.slice/crio-c71edb6cb2d7091136d43826ece3cc92e6c185e45dbf3043fa164e28b2d801b7 WatchSource:0}: Error finding container c71edb6cb2d7091136d43826ece3cc92e6c185e45dbf3043fa164e28b2d801b7: Status 404 returned error can't find the container with id c71edb6cb2d7091136d43826ece3cc92e6c185e45dbf3043fa164e28b2d801b7
Sep 17 15:56:16.593626 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:16.593596    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ql9hb" event={"ID":"d9162039-1eba-40eb-af07-47697bc0cc05","Type":"ContainerStarted","Data":"c71edb6cb2d7091136d43826ece3cc92e6c185e45dbf3043fa164e28b2d801b7"}
Sep 17 15:56:17.598875 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:17.598842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ql9hb" event={"ID":"d9162039-1eba-40eb-af07-47697bc0cc05","Type":"ContainerStarted","Data":"88cf1f3ccffde73ff4c407c25a90d5f165b4f18e37429d59a923b67cd8baec63"}
Sep 17 15:56:17.599228 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:17.598898    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ql9hb" event={"ID":"d9162039-1eba-40eb-af07-47697bc0cc05","Type":"ContainerStarted","Data":"2f5fc0365b922b4b9f14451d314f2c538235702e2c1a40005d5595c2c6fa74a5"}
Sep 17 15:56:18.603519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:18.603496    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ql9hb" event={"ID":"d9162039-1eba-40eb-af07-47697bc0cc05","Type":"ContainerStarted","Data":"cd76905655ad47b4d2476f531dae54e5ee86c12930ca5f9283f9413dad96fee8"}
Sep 17 15:56:18.627903 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:18.627858    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ql9hb" podStartSLOduration=16.736028372 podStartE2EDuration="18.62784263s" totalImagesPullingTime="1.891814258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:56:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:18.627326108 +0000 UTC m=+128.064002888" watchObservedRunningTime="2026-09-17 15:56:18.62784263 +0000 UTC m=+128.064519405"
Sep 17 15:56:19.040066 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.040035    2566 scope.go:117] "RemoveContainer" containerID="c47444815e7aab3e4ffe3d82bfa971f599a5ed7607b9536cab09cf7fe94f04bb"
Sep 17 15:56:19.609889 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.609862    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 15:56:19.610332 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.610318    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/1.log"
Sep 17 15:56:19.610378 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.610350    2566 generic.go:356] "Generic (PLEG): container finished" podID="6c520bd2-3078-4ef1-a685-70e71b9467af" containerID="3863e699ca4418e741d745e5ffb966e10d2016552f9d2aea70da364817df9a2b" exitCode=255
Sep 17 15:56:19.610440 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.610421    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" event={"ID":"6c520bd2-3078-4ef1-a685-70e71b9467af","Type":"ContainerDied","Data":"3863e699ca4418e741d745e5ffb966e10d2016552f9d2aea70da364817df9a2b"}
Sep 17 15:56:19.610479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.610474    2566 scope.go:117] "RemoveContainer" containerID="c47444815e7aab3e4ffe3d82bfa971f599a5ed7607b9536cab09cf7fe94f04bb"
Sep 17 15:56:19.610862 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.610840    2566 scope.go:117] "RemoveContainer" containerID="3863e699ca4418e741d745e5ffb966e10d2016552f9d2aea70da364817df9a2b"
Sep 17 15:56:19.611010 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:19.610992    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-fd4bb46c-2zp6f_openshift-console-operator(6c520bd2-3078-4ef1-a685-70e71b9467af)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" podUID="6c520bd2-3078-4ef1-a685-70e71b9467af"
Sep 17 15:56:19.773268 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.773229    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:56:19.775401 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:19.775375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f8edd7-4332-49c1-8f85-00a63475c7ed-metrics-certs\") pod \"network-metrics-daemon-vvjn8\" (UID: \"26f8edd7-4332-49c1-8f85-00a63475c7ed\") " pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:56:20.060197 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:20.060152    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8v7v5\""
Sep 17 15:56:20.068376 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:20.068356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vvjn8"
Sep 17 15:56:20.348452 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:20.348379    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vvjn8"]
Sep 17 15:56:20.351326 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:20.351301    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f8edd7_4332_49c1_8f85_00a63475c7ed.slice/crio-ab3469da908d84ab1aed2bdf1a0b1582dfa9dd99912519e06fcbe503c6200ad5 WatchSource:0}: Error finding container ab3469da908d84ab1aed2bdf1a0b1582dfa9dd99912519e06fcbe503c6200ad5: Status 404 returned error can't find the container with id ab3469da908d84ab1aed2bdf1a0b1582dfa9dd99912519e06fcbe503c6200ad5
Sep 17 15:56:20.614602 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:20.614517    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vvjn8" event={"ID":"26f8edd7-4332-49c1-8f85-00a63475c7ed","Type":"ContainerStarted","Data":"ab3469da908d84ab1aed2bdf1a0b1582dfa9dd99912519e06fcbe503c6200ad5"}
Sep 17 15:56:20.615917 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:20.615895    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 15:56:21.621720 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:21.621691    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vvjn8" event={"ID":"26f8edd7-4332-49c1-8f85-00a63475c7ed","Type":"ContainerStarted","Data":"44784454877c75a06b0f2023d4746f6f64649cc7fc7de1fd6b02bfdab4f379c3"}
Sep 17 15:56:22.627309 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:22.627274    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vvjn8" event={"ID":"26f8edd7-4332-49c1-8f85-00a63475c7ed","Type":"ContainerStarted","Data":"387a1d66756228ca27b1fdd443ad3ef64b44c5df4a66b35e6b59dd40a942fd7d"}
Sep 17 15:56:22.673705 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:22.673650    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vvjn8" podStartSLOduration=129.578837129 podStartE2EDuration="2m10.673629057s" totalImagesPullingTime="1.094791928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:22.672493871 +0000 UTC m=+132.109170649" watchObservedRunningTime="2026-09-17 15:56:22.673629057 +0000 UTC m=+132.110305818"
Sep 17 15:56:26.348592 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:26.348559    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:56:26.348951 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:26.348611    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:56:26.348951 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:26.348901    2566 scope.go:117] "RemoveContainer" containerID="3863e699ca4418e741d745e5ffb966e10d2016552f9d2aea70da364817df9a2b"
Sep 17 15:56:26.349080 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:26.349060    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-fd4bb46c-2zp6f_openshift-console-operator(6c520bd2-3078-4ef1-a685-70e71b9467af)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" podUID="6c520bd2-3078-4ef1-a685-70e71b9467af"
Sep 17 15:56:32.146854 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:32.146822    2566 patch_prober.go:28] interesting pod/image-registry-66cf577dcf-2wh9p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 15:56:32.147327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:32.146877    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" podUID="2ab36b25-1767-452f-8b74-520b39589954" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:56:33.582383 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:33.582351    2566 patch_prober.go:28] interesting pod/image-registry-66cf577dcf-2wh9p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 15:56:33.582881 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:33.582407    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" podUID="2ab36b25-1767-452f-8b74-520b39589954" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:56:36.948370 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:36.948337    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"]
Sep 17 15:56:36.950493 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:36.950476    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:36.956748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:36.956726    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 17 15:56:36.956856 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:36.956812    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fs2vx\""
Sep 17 15:56:36.956906 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:36.956851    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 17 15:56:36.981233 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:36.981205    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"]
Sep 17 15:56:37.008494 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.008469    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66cf577dcf-2wh9p"]
Sep 17 15:56:37.012515 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.012487    2566 patch_prober.go:28] interesting pod/image-registry-66cf577dcf-2wh9p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 15:56:37.012620 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.012523    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" podUID="2ab36b25-1767-452f-8b74-520b39589954" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 15:56:37.086068 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.086038    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e76500bb-5ea7-4268-b949-aa1b8266e81f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-4vjtr\" (UID: \"e76500bb-5ea7-4268-b949-aa1b8266e81f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:37.086197 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.086098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e76500bb-5ea7-4268-b949-aa1b8266e81f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-4vjtr\" (UID: \"e76500bb-5ea7-4268-b949-aa1b8266e81f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:37.086848 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.086824    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"]
Sep 17 15:56:37.088806 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.088792    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"
Sep 17 15:56:37.095946 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.095925    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 17 15:56:37.099552 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.099534    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sdn9d\""
Sep 17 15:56:37.133761 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.133738    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"]
Sep 17 15:56:37.185051 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.185031    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c6965dbd6-kpr7x"]
Sep 17 15:56:37.186984 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.186969    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.187126 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.187108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e76500bb-5ea7-4268-b949-aa1b8266e81f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-4vjtr\" (UID: \"e76500bb-5ea7-4268-b949-aa1b8266e81f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:37.187179 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.187147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e76500bb-5ea7-4268-b949-aa1b8266e81f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-4vjtr\" (UID: \"e76500bb-5ea7-4268-b949-aa1b8266e81f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:37.187225 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.187189    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0b27a7d5-5dd2-4cbf-9abc-c5642495ae31-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-jj4gl\" (UID: \"0b27a7d5-5dd2-4cbf-9abc-c5642495ae31\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"
Sep 17 15:56:37.187722 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.187700    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e76500bb-5ea7-4268-b949-aa1b8266e81f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-4vjtr\" (UID: \"e76500bb-5ea7-4268-b949-aa1b8266e81f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:37.189465 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.189449    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e76500bb-5ea7-4268-b949-aa1b8266e81f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-4vjtr\" (UID: \"e76500bb-5ea7-4268-b949-aa1b8266e81f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:37.222743 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.222694    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6965dbd6-kpr7x"]
Sep 17 15:56:37.259192 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.259149    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"
Sep 17 15:56:37.287952 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.287924    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-registry-tls\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.288081 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.287963    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfjff\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-kube-api-access-xfjff\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.288081 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.288003    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edae5621-2635-4270-bdf1-c6900c53f3a7-registry-certificates\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.288081 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.288030    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edae5621-2635-4270-bdf1-c6900c53f3a7-installation-pull-secrets\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.288081 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.288062    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edae5621-2635-4270-bdf1-c6900c53f3a7-ca-trust-extracted\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.288265 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.288098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-bound-sa-token\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.288265 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.288214    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edae5621-2635-4270-bdf1-c6900c53f3a7-trusted-ca\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.288265 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.288244    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0b27a7d5-5dd2-4cbf-9abc-c5642495ae31-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-jj4gl\" (UID: \"0b27a7d5-5dd2-4cbf-9abc-c5642495ae31\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"
Sep 17 15:56:37.288396 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.288266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edae5621-2635-4270-bdf1-c6900c53f3a7-image-registry-private-configuration\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.290687 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.290659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0b27a7d5-5dd2-4cbf-9abc-c5642495ae31-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-jj4gl\" (UID: \"0b27a7d5-5dd2-4cbf-9abc-c5642495ae31\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"
Sep 17 15:56:37.388580 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388551    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edae5621-2635-4270-bdf1-c6900c53f3a7-registry-certificates\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.388580 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edae5621-2635-4270-bdf1-c6900c53f3a7-installation-pull-secrets\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.388748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388603    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edae5621-2635-4270-bdf1-c6900c53f3a7-ca-trust-extracted\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.388748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388633    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-bound-sa-token\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.388748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edae5621-2635-4270-bdf1-c6900c53f3a7-trusted-ca\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.388748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edae5621-2635-4270-bdf1-c6900c53f3a7-image-registry-private-configuration\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.388748 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-registry-tls\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.388981 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.388760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfjff\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-kube-api-access-xfjff\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.389354 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.389300    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edae5621-2635-4270-bdf1-c6900c53f3a7-ca-trust-extracted\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.389519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.389457    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edae5621-2635-4270-bdf1-c6900c53f3a7-registry-certificates\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.389801 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.389778    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edae5621-2635-4270-bdf1-c6900c53f3a7-trusted-ca\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.391060 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.391039    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edae5621-2635-4270-bdf1-c6900c53f3a7-installation-pull-secrets\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.391232 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.391212    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edae5621-2635-4270-bdf1-c6900c53f3a7-image-registry-private-configuration\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.391280 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.391212    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-registry-tls\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.397068 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.397051    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"
Sep 17 15:56:37.412812 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.412790    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfjff\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-kube-api-access-xfjff\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.418501 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.416896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edae5621-2635-4270-bdf1-c6900c53f3a7-bound-sa-token\") pod \"image-registry-c6965dbd6-kpr7x\" (UID: \"edae5621-2635-4270-bdf1-c6900c53f3a7\") " pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.500259 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.500236    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:37.591043 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.591013    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr"]
Sep 17 15:56:37.593657 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:37.593630    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode76500bb_5ea7_4268_b949_aa1b8266e81f.slice/crio-3f515c5e391296778f5337a37cf490bfae31c3e0a8786d310c7f60ac545c5c7e WatchSource:0}: Error finding container 3f515c5e391296778f5337a37cf490bfae31c3e0a8786d310c7f60ac545c5c7e: Status 404 returned error can't find the container with id 3f515c5e391296778f5337a37cf490bfae31c3e0a8786d310c7f60ac545c5c7e
Sep 17 15:56:37.673785 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.673751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr" event={"ID":"e76500bb-5ea7-4268-b949-aa1b8266e81f","Type":"ContainerStarted","Data":"3f515c5e391296778f5337a37cf490bfae31c3e0a8786d310c7f60ac545c5c7e"}
Sep 17 15:56:37.697050 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.697031    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"]
Sep 17 15:56:37.699306 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:37.699285    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b27a7d5_5dd2_4cbf_9abc_c5642495ae31.slice/crio-2f0c17ca5ca04632b9cc6ad8744a7704799fbce7a9d2c9606f0b3c7b2e8b1021 WatchSource:0}: Error finding container 2f0c17ca5ca04632b9cc6ad8744a7704799fbce7a9d2c9606f0b3c7b2e8b1021: Status 404 returned error can't find the container with id 2f0c17ca5ca04632b9cc6ad8744a7704799fbce7a9d2c9606f0b3c7b2e8b1021
Sep 17 15:56:37.801966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:37.801943    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6965dbd6-kpr7x"]
Sep 17 15:56:37.804042 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:37.804022    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedae5621_2635_4270_bdf1_c6900c53f3a7.slice/crio-bfe691017eb990393922edf5556b479911586c91de50887fe26295d90c5c180b WatchSource:0}: Error finding container bfe691017eb990393922edf5556b479911586c91de50887fe26295d90c5c180b: Status 404 returned error can't find the container with id bfe691017eb990393922edf5556b479911586c91de50887fe26295d90c5c180b
Sep 17 15:56:38.678333 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:38.678301    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x" event={"ID":"edae5621-2635-4270-bdf1-c6900c53f3a7","Type":"ContainerStarted","Data":"bc30682fa0b93e5271509eb54037e6a6b77978cbd839060e09b9e09707e2eb46"}
Sep 17 15:56:38.678850 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:38.678352    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x" event={"ID":"edae5621-2635-4270-bdf1-c6900c53f3a7","Type":"ContainerStarted","Data":"bfe691017eb990393922edf5556b479911586c91de50887fe26295d90c5c180b"}
Sep 17 15:56:38.678850 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:38.678412    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:38.679685 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:38.679661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl" event={"ID":"0b27a7d5-5dd2-4cbf-9abc-c5642495ae31","Type":"ContainerStarted","Data":"2f0c17ca5ca04632b9cc6ad8744a7704799fbce7a9d2c9606f0b3c7b2e8b1021"}
Sep 17 15:56:39.684614 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:39.684577    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl" event={"ID":"0b27a7d5-5dd2-4cbf-9abc-c5642495ae31","Type":"ContainerStarted","Data":"1d2f124d66e3482338da67a94dbc361e266ab685161890ba96594a03fe94596a"}
Sep 17 15:56:39.685061 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:39.684769    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"
Sep 17 15:56:39.686197 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:39.686152    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr" event={"ID":"e76500bb-5ea7-4268-b949-aa1b8266e81f","Type":"ContainerStarted","Data":"7f692d7cef015c914c4b3e2c0446022494352114a349ba9d558f0331c68ef6fc"}
Sep 17 15:56:39.689530 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:39.689511    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl"
Sep 17 15:56:39.703853 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:39.703807    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-jj4gl" podStartSLOduration=2.451669355 podStartE2EDuration="3.703794089s" totalImagesPullingTime="1.252124734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:39.703141664 +0000 UTC m=+149.139818444" watchObservedRunningTime="2026-09-17 15:56:39.703794089 +0000 UTC m=+149.140470868"
Sep 17 15:56:39.704065 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:39.704038    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x" podStartSLOduration=2.70402861 podStartE2EDuration="2.70402861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:38.704646955 +0000 UTC m=+148.141323735" watchObservedRunningTime="2026-09-17 15:56:39.70402861 +0000 UTC m=+149.140705382"
Sep 17 15:56:39.721221 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:39.721181    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-4vjtr" podStartSLOduration=2.365702544 podStartE2EDuration="3.721150948s" totalImagesPullingTime="1.355448404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:39.720058124 +0000 UTC m=+149.156734906" watchObservedRunningTime="2026-09-17 15:56:39.721150948 +0000 UTC m=+149.157827761"
Sep 17 15:56:41.040855 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:41.040818    2566 scope.go:117] "RemoveContainer" containerID="3863e699ca4418e741d745e5ffb966e10d2016552f9d2aea70da364817df9a2b"
Sep 17 15:56:41.695097 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:41.695069    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 15:56:41.695272 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:41.695129    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" event={"ID":"6c520bd2-3078-4ef1-a685-70e71b9467af","Type":"ContainerStarted","Data":"81f61614537fae21bd3532e9ae59cd4984feac41e016a1e7a7331d59839c5c1a"}
Sep 17 15:56:41.695477 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:41.695448    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:56:41.700081 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:41.700053    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f"
Sep 17 15:56:41.719072 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:41.719033    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-2zp6f" podStartSLOduration=44.097698188 podStartE2EDuration="46.719020555s" totalImagesPullingTime="2.621322367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:41.718599417 +0000 UTC m=+151.155276198" watchObservedRunningTime="2026-09-17 15:56:41.719020555 +0000 UTC m=+151.155697358"
Sep 17 15:56:44.933173 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:44.933128    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"]
Sep 17 15:56:44.938503 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:44.938485    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:44.941348 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:44.941324    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 15:56:44.942441 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:44.942425    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 15:56:44.942441 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:44.942434    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ldnph\""
Sep 17 15:56:44.942537 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:44.942496    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 15:56:44.947122 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:44.947103    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"]
Sep 17 15:56:45.041850 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.041812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.042007 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.041864    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.042007 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.041974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bb66\" (UniqueName: \"kubernetes.io/projected/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-kube-api-access-6bb66\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.042187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.042067    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.044452 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.044434    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wzd5l"]
Sep 17 15:56:45.049189 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.049174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.051575 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.051556    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dc5q2\""
Sep 17 15:56:45.051924 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.051906    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 15:56:45.052352 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.052337    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 15:56:45.053367 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.053349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 15:56:45.142694 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-tls\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.142823 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142704    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.142823 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142724    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-root\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.142823 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142742    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-accelerators-collector-config\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.142964 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2hsf\" (UniqueName: \"kubernetes.io/projected/d5947eed-4564-4108-8b06-33222fe68cc4-kube-api-access-z2hsf\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.142964 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-wtmp\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.142964 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bb66\" (UniqueName: \"kubernetes.io/projected/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-kube-api-access-6bb66\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.142964 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.143179 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.142978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5947eed-4564-4108-8b06-33222fe68cc4-metrics-client-ca\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.143179 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.143020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.143179 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.143046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-sys\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.143179 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.143093    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.143179 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:45.143125    2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 17 15:56:45.143179 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.143137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-textfile\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.143421 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:45.143205    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-tls podName:2d2fd29c-0300-4d5b-8c09-36c7a29f3e93 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:45.64318728 +0000 UTC m=+155.079864041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-pfqf6" (UID: "2d2fd29c-0300-4d5b-8c09-36c7a29f3e93") : secret "openshift-state-metrics-tls" not found
Sep 17 15:56:45.143555 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.143528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.145357 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.145340    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.157974 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.157956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bb66\" (UniqueName: \"kubernetes.io/projected/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-kube-api-access-6bb66\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.244016 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.243993    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2hsf\" (UniqueName: \"kubernetes.io/projected/d5947eed-4564-4108-8b06-33222fe68cc4-kube-api-access-z2hsf\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244112 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-wtmp\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244112 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244112 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244061    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5947eed-4564-4108-8b06-33222fe68cc4-metrics-client-ca\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244280 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-sys\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244280 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244202    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-wtmp\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244280 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244255    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-sys\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244280 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244257    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-textfile\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244442 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-tls\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244442 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244348    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-root\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244442 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-accelerators-collector-config\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244442 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244426    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5947eed-4564-4108-8b06-33222fe68cc4-root\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244637 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244477    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-textfile\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244637 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244631    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5947eed-4564-4108-8b06-33222fe68cc4-metrics-client-ca\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.244809 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.244792    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-accelerators-collector-config\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.246254 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.246235    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.246368 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.246349    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5947eed-4564-4108-8b06-33222fe68cc4-node-exporter-tls\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.261400 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.261376    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2hsf\" (UniqueName: \"kubernetes.io/projected/d5947eed-4564-4108-8b06-33222fe68cc4-kube-api-access-z2hsf\") pod \"node-exporter-wzd5l\" (UID: \"d5947eed-4564-4108-8b06-33222fe68cc4\") " pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.357970 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.357939    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wzd5l"
Sep 17 15:56:45.370200 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:45.370172    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5947eed_4564_4108_8b06_33222fe68cc4.slice/crio-dc24b7acf7836c66d36faf4c68c048c63d72e88f35ae4351b7f4d634b1fe7809 WatchSource:0}: Error finding container dc24b7acf7836c66d36faf4c68c048c63d72e88f35ae4351b7f4d634b1fe7809: Status 404 returned error can't find the container with id dc24b7acf7836c66d36faf4c68c048c63d72e88f35ae4351b7f4d634b1fe7809
Sep 17 15:56:45.647626 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.647543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.649999 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.649962    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2fd29c-0300-4d5b-8c09-36c7a29f3e93-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pfqf6\" (UID: \"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:45.711619 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.711593    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzd5l" event={"ID":"d5947eed-4564-4108-8b06-33222fe68cc4","Type":"ContainerStarted","Data":"dc24b7acf7836c66d36faf4c68c048c63d72e88f35ae4351b7f4d634b1fe7809"}
Sep 17 15:56:45.848270 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:45.848245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"
Sep 17 15:56:46.063764 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.063725    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:56:46.068480 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.068451    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.076923 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.076900    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 15:56:46.076923 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.076902    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 15:56:46.077101 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.076953    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4c7sb\""
Sep 17 15:56:46.077101 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.076971    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 15:56:46.077345 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.077327    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 15:56:46.078269 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.078249    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 15:56:46.078552 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.078533    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 15:56:46.078633 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.078541    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 15:56:46.078820 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.078801    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 15:56:46.078966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.078945    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 15:56:46.090205 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.090181    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:56:46.138095 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.138004    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6"]
Sep 17 15:56:46.140454 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:46.140430    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d2fd29c_0300_4d5b_8c09_36c7a29f3e93.slice/crio-08b335aa5ca024877ccc5d82dcefc20f8f64b5796eb53137055b166c2872a603 WatchSource:0}: Error finding container 08b335aa5ca024877ccc5d82dcefc20f8f64b5796eb53137055b166c2872a603: Status 404 returned error can't find the container with id 08b335aa5ca024877ccc5d82dcefc20f8f64b5796eb53137055b166c2872a603
Sep 17 15:56:46.151500 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-web-config\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151610 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151610 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151717 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151621    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151717 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151660    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-volume\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151717 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151684    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151717 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151713    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-out\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151911 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151753    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151911 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151788    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151911 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.151911 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nd5k\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-kube-api-access-8nd5k\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.152087 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.152087 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.151988    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253082 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-out\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253263 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253263 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253263 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253179    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253263 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nd5k\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-kube-api-access-8nd5k\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253276    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253345    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-web-config\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253409    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253437    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253479 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253470    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-volume\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253809 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.253502    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.253809 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:46.253613    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 17 15:56:46.253809 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:46.253680    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls podName:da4b96de-fe9d-4eef-8384-ebe5b996be26 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:46.753660905 +0000 UTC m=+156.190337662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26") : secret "alertmanager-main-tls" not found
Sep 17 15:56:46.254930 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.254100    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.254930 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.254418    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.254930 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.254874    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.256298 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.256272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-out\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.256707 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.256684    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-web-config\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.257949 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.257385    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.257949 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.257607    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.257949 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.257637    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.257949 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.257902    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.258714 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.258696    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-volume\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.259061 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.259035    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.264326 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.264299    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nd5k\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-kube-api-access-8nd5k\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.716979 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.716944    2566 generic.go:356] "Generic (PLEG): container finished" podID="d5947eed-4564-4108-8b06-33222fe68cc4" containerID="7597eeb2f9b88a6b22a18eed884cff8e6efd029c8635ef5bb26a7290ca342aa9" exitCode=0
Sep 17 15:56:46.717129 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.717026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzd5l" event={"ID":"d5947eed-4564-4108-8b06-33222fe68cc4","Type":"ContainerDied","Data":"7597eeb2f9b88a6b22a18eed884cff8e6efd029c8635ef5bb26a7290ca342aa9"}
Sep 17 15:56:46.719146 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.719123    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6" event={"ID":"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93","Type":"ContainerStarted","Data":"269427fccb5862c7d81764d7f7a5753987548ddbef6f6e60ada3cdce03b1655c"}
Sep 17 15:56:46.719249 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.719200    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6" event={"ID":"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93","Type":"ContainerStarted","Data":"1b844a15dcbf46adea176261300842274e2cd225ab1a8d8d3cf823cf0e9a9698"}
Sep 17 15:56:46.719249 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.719231    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6" event={"ID":"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93","Type":"ContainerStarted","Data":"08b335aa5ca024877ccc5d82dcefc20f8f64b5796eb53137055b166c2872a603"}
Sep 17 15:56:46.756344 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.756323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.758573 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.758547    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:46.980519 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:46.980485    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:56:47.013727 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.013667    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:56:47.301433 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.301351    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:56:47.304244 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:47.304209    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4b96de_fe9d_4eef_8384_ebe5b996be26.slice/crio-141dc05b164c8678b9beeae6f0637911663c5a647f0088ea645b79a3ba870862 WatchSource:0}: Error finding container 141dc05b164c8678b9beeae6f0637911663c5a647f0088ea645b79a3ba870862: Status 404 returned error can't find the container with id 141dc05b164c8678b9beeae6f0637911663c5a647f0088ea645b79a3ba870862
Sep 17 15:56:47.724905 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.724864    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzd5l" event={"ID":"d5947eed-4564-4108-8b06-33222fe68cc4","Type":"ContainerStarted","Data":"3047ce76c106e04193264f9a3defd368e42fadb18a73e2ccb8d401d383bb8da1"}
Sep 17 15:56:47.725058 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.724969    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzd5l" event={"ID":"d5947eed-4564-4108-8b06-33222fe68cc4","Type":"ContainerStarted","Data":"88fcea6b2540ab7c7cd9cde2fddf64fbf313146f4a07d795b11afe1b96c6e1ad"}
Sep 17 15:56:47.727104 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.727080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6" event={"ID":"2d2fd29c-0300-4d5b-8c09-36c7a29f3e93","Type":"ContainerStarted","Data":"578e0b2afc95ccee3f4d97f7ac9997941dcb38f2ab5e51e739b93ca50d3a9077"}
Sep 17 15:56:47.728233 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.728210    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerStarted","Data":"141dc05b164c8678b9beeae6f0637911663c5a647f0088ea645b79a3ba870862"}
Sep 17 15:56:47.766919 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.766870    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wzd5l" podStartSLOduration=1.6455473550000002 podStartE2EDuration="2.766856491s" totalImagesPullingTime="1.058386614s" totalInitContainerRuntime="62.922522ms" isStatefulPod=true podCreationTimestamp="2026-09-17 15:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:47.766389263 +0000 UTC m=+157.203066042" watchObservedRunningTime="2026-09-17 15:56:47.766856491 +0000 UTC m=+157.203533270"
Sep 17 15:56:47.798782 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:47.798739    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pfqf6" podStartSLOduration=2.745200918 podStartE2EDuration="3.798728316s" totalImagesPullingTime="1.053527398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:47.798590197 +0000 UTC m=+157.235266991" watchObservedRunningTime="2026-09-17 15:56:47.798728316 +0000 UTC m=+157.235405095"
Sep 17 15:56:48.185741 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.185709    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"]
Sep 17 15:56:48.189680 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.189660    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.194329 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.194308    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bhat56gsje54d\""
Sep 17 15:56:48.194461 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.194383    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 17 15:56:48.194690 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.194669    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 17 15:56:48.194941 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.194918    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 17 15:56:48.195029 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.194961    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zm57r\""
Sep 17 15:56:48.195083 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.194961    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 17 15:56:48.195296 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.195207    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 17 15:56:48.225678 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.225657    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"]
Sep 17 15:56:48.271851 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.271811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.271975 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.271879    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.271975 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.271919    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-tls\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.272085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.271987    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.272085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.272031    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.272196 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.272090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-grpc-tls\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.272196 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.272114    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcc2\" (UniqueName: \"kubernetes.io/projected/f72c4aee-2ba9-479b-916a-f60b46960d41-kube-api-access-btcc2\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.272196 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.272148    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72c4aee-2ba9-479b-916a-f60b46960d41-metrics-client-ca\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.372728 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.372694    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.372772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-grpc-tls\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.372803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcc2\" (UniqueName: \"kubernetes.io/projected/f72c4aee-2ba9-479b-916a-f60b46960d41-kube-api-access-btcc2\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.372841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72c4aee-2ba9-479b-916a-f60b46960d41-metrics-client-ca\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.372972    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.373039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.373078    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-tls\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373177 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.373123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.373637 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.373615    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72c4aee-2ba9-479b-916a-f60b46960d41-metrics-client-ca\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.376042 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.375965    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.376042 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.375974    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-grpc-tls\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.376229 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.376093    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.376333 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.376312    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-tls\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.376649 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.376624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.376858 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.376838    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f72c4aee-2ba9-479b-916a-f60b46960d41-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.383486 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.383464    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcc2\" (UniqueName: \"kubernetes.io/projected/f72c4aee-2ba9-479b-916a-f60b46960d41-kube-api-access-btcc2\") pod \"thanos-querier-9f5cb6d45-scmg9\" (UID: \"f72c4aee-2ba9-479b-916a-f60b46960d41\") " pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.500737 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.500712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:48.733092 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.733014    2566 generic.go:356] "Generic (PLEG): container finished" podID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerID="1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c" exitCode=0
Sep 17 15:56:48.733263 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.733101    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c"}
Sep 17 15:56:48.784625 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:48.784599    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"]
Sep 17 15:56:48.787140 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:48.787116    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72c4aee_2ba9_479b_916a_f60b46960d41.slice/crio-92b1196504dfe3eb81520e9ff2b58b6c7a8eef1b2360e04d033d90bdaab12601 WatchSource:0}: Error finding container 92b1196504dfe3eb81520e9ff2b58b6c7a8eef1b2360e04d033d90bdaab12601: Status 404 returned error can't find the container with id 92b1196504dfe3eb81520e9ff2b58b6c7a8eef1b2360e04d033d90bdaab12601
Sep 17 15:56:49.048778 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:49.048745    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6qjsh" podUID="0be3ee88-c394-4754-8569-fde1369da3e1"
Sep 17 15:56:49.692572 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.692390    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"]
Sep 17 15:56:49.697005 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.696981    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:49.699561 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.699538    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 17 15:56:49.699842 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.699827    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xjxrh\""
Sep 17 15:56:49.709701 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.709646    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"]
Sep 17 15:56:49.744839 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.744750    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" event={"ID":"f72c4aee-2ba9-479b-916a-f60b46960d41","Type":"ContainerStarted","Data":"92b1196504dfe3eb81520e9ff2b58b6c7a8eef1b2360e04d033d90bdaab12601"}
Sep 17 15:56:49.744839 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.744782    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:56:49.785782 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.785735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zmr9\" (UniqueName: \"kubernetes.io/projected/8cbca560-0e32-447b-a0ee-5c2280afca4a-kube-api-access-5zmr9\") pod \"monitoring-plugin-5984459ff8-f58fr\" (UID: \"8cbca560-0e32-447b-a0ee-5c2280afca4a\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:49.785896 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.785791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cbca560-0e32-447b-a0ee-5c2280afca4a-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-f58fr\" (UID: \"8cbca560-0e32-447b-a0ee-5c2280afca4a\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:49.886452 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.886415    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zmr9\" (UniqueName: \"kubernetes.io/projected/8cbca560-0e32-447b-a0ee-5c2280afca4a-kube-api-access-5zmr9\") pod \"monitoring-plugin-5984459ff8-f58fr\" (UID: \"8cbca560-0e32-447b-a0ee-5c2280afca4a\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:49.886625 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.886474    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cbca560-0e32-447b-a0ee-5c2280afca4a-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-f58fr\" (UID: \"8cbca560-0e32-447b-a0ee-5c2280afca4a\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:49.886625 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:49.886583    2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 17 15:56:49.886742 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:56:49.886657    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbca560-0e32-447b-a0ee-5c2280afca4a-monitoring-plugin-cert podName:8cbca560-0e32-447b-a0ee-5c2280afca4a nodeName:}" failed. No retries permitted until 2026-09-17 15:56:50.386635101 +0000 UTC m=+159.823311862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8cbca560-0e32-447b-a0ee-5c2280afca4a-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-f58fr" (UID: "8cbca560-0e32-447b-a0ee-5c2280afca4a") : secret "monitoring-plugin-cert" not found
Sep 17 15:56:49.898450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:49.898424    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zmr9\" (UniqueName: \"kubernetes.io/projected/8cbca560-0e32-447b-a0ee-5c2280afca4a-kube-api-access-5zmr9\") pod \"monitoring-plugin-5984459ff8-f58fr\" (UID: \"8cbca560-0e32-447b-a0ee-5c2280afca4a\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:50.391734 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.391704    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cbca560-0e32-447b-a0ee-5c2280afca4a-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-f58fr\" (UID: \"8cbca560-0e32-447b-a0ee-5c2280afca4a\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:50.394449 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.394422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cbca560-0e32-447b-a0ee-5c2280afca4a-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-f58fr\" (UID: \"8cbca560-0e32-447b-a0ee-5c2280afca4a\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:50.608417 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.608337    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:50.752679 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.752638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerStarted","Data":"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"}
Sep 17 15:56:50.753085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.752706    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerStarted","Data":"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"}
Sep 17 15:56:50.753085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.752740    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerStarted","Data":"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"}
Sep 17 15:56:50.753085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.752777    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerStarted","Data":"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"}
Sep 17 15:56:50.753085 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.752801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerStarted","Data":"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"}
Sep 17 15:56:50.953768 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.952618    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fb8f454b8-rj8db"]
Sep 17 15:56:50.956398 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.956371    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:50.961957 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.961754    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 15:56:50.961957 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.961801    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 15:56:50.961957 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.961819    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 15:56:50.962405 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.962062    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 15:56:50.963885 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.963106    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 15:56:50.963885 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.963172    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 15:56:50.963885 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.963176    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7p6v8\""
Sep 17 15:56:50.963885 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.963212    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 15:56:50.968974 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:50.968958    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb8f454b8-rj8db"]
Sep 17 15:56:51.098073 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.098043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhkbh\" (UniqueName: \"kubernetes.io/projected/ee215151-aa4d-4df0-813a-61f9eef4e964-kube-api-access-bhkbh\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.098211 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.098089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-console-config\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.098211 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.098123    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-serving-cert\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.098318 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.098208    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-service-ca\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.098318 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.098266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-oauth-config\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.098421 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.098345    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-oauth-serving-cert\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.179703 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.179650    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"]
Sep 17 15:56:51.182230 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:51.182199    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbca560_0e32_447b_a0ee_5c2280afca4a.slice/crio-6fac88d3b69b4610d40ad9101fafe89586f6871ac062470818478848732ba93b WatchSource:0}: Error finding container 6fac88d3b69b4610d40ad9101fafe89586f6871ac062470818478848732ba93b: Status 404 returned error can't find the container with id 6fac88d3b69b4610d40ad9101fafe89586f6871ac062470818478848732ba93b
Sep 17 15:56:51.199817 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.199793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-service-ca\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.199911 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.199847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-oauth-config\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.199953 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.199922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-oauth-serving-cert\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.199985 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.199971    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhkbh\" (UniqueName: \"kubernetes.io/projected/ee215151-aa4d-4df0-813a-61f9eef4e964-kube-api-access-bhkbh\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.200021 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.199999    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-console-config\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.200053 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.200032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-serving-cert\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.201089 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.200799    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-console-config\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.201089 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.200851    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-service-ca\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.201089 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.201015    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-oauth-serving-cert\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.202528 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.202508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-serving-cert\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.202528 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.202522    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-oauth-config\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.209811 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.209789    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhkbh\" (UniqueName: \"kubernetes.io/projected/ee215151-aa4d-4df0-813a-61f9eef4e964-kube-api-access-bhkbh\") pod \"console-fb8f454b8-rj8db\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") " pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.268939 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.268897    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:56:51.608303 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.608272    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb8f454b8-rj8db"]
Sep 17 15:56:51.611276 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:51.611247    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee215151_aa4d_4df0_813a_61f9eef4e964.slice/crio-bab9b5163dac6003ff0a244c318aea1c81242341874ba6c0ca3d4345b25b5cdd WatchSource:0}: Error finding container bab9b5163dac6003ff0a244c318aea1c81242341874ba6c0ca3d4345b25b5cdd: Status 404 returned error can't find the container with id bab9b5163dac6003ff0a244c318aea1c81242341874ba6c0ca3d4345b25b5cdd
Sep 17 15:56:51.760627 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.760588    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" event={"ID":"f72c4aee-2ba9-479b-916a-f60b46960d41","Type":"ContainerStarted","Data":"414b0348bd046309e233e4c9c4c8744fe4db7ee8cf5eb7c56c02be8d9cb32140"}
Sep 17 15:56:51.761057 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.760695    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" event={"ID":"f72c4aee-2ba9-479b-916a-f60b46960d41","Type":"ContainerStarted","Data":"477bb7e4e445fa9a6d8974a91d552bf6950e5f901ecce39cd1d662e4283ba175"}
Sep 17 15:56:51.761057 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.760738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" event={"ID":"f72c4aee-2ba9-479b-916a-f60b46960d41","Type":"ContainerStarted","Data":"8e9c4e4d35dd8c4135075f8aa05b5d749403b275c71487d129667b19174a5346"}
Sep 17 15:56:51.762118 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.762096    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb8f454b8-rj8db" event={"ID":"ee215151-aa4d-4df0-813a-61f9eef4e964","Type":"ContainerStarted","Data":"bab9b5163dac6003ff0a244c318aea1c81242341874ba6c0ca3d4345b25b5cdd"}
Sep 17 15:56:51.767279 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.767248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerStarted","Data":"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"}
Sep 17 15:56:51.774739 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.774656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr" event={"ID":"8cbca560-0e32-447b-a0ee-5c2280afca4a","Type":"ContainerStarted","Data":"6fac88d3b69b4610d40ad9101fafe89586f6871ac062470818478848732ba93b"}
Sep 17 15:56:51.806657 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:51.806620    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.319173032 podStartE2EDuration="5.806605365s" totalImagesPullingTime="3.471070456s" totalInitContainerRuntime="16.361877ms" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:51.803763226 +0000 UTC m=+161.240440000" watchObservedRunningTime="2026-09-17 15:56:51.806605365 +0000 UTC m=+161.243282144"
Sep 17 15:56:52.789087 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:52.788905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" event={"ID":"f72c4aee-2ba9-479b-916a-f60b46960d41","Type":"ContainerStarted","Data":"eafa2411e72b318fa7db1e678bd615438af9bc337f4a73af916cbe88e991a473"}
Sep 17 15:56:52.789087 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:52.788995    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" event={"ID":"f72c4aee-2ba9-479b-916a-f60b46960d41","Type":"ContainerStarted","Data":"da977d66dcd23dea728d254c642603536ca0567fad65d5ba539fe439ac75eb90"}
Sep 17 15:56:52.789087 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:52.789038    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" event={"ID":"f72c4aee-2ba9-479b-916a-f60b46960d41","Type":"ContainerStarted","Data":"832cc3befa1bc3a9011f4b31d449f37bc86f84638a6a949938e887d1856608a0"}
Sep 17 15:56:52.820381 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:52.820331    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9" podStartSLOduration=1.998106607 podStartE2EDuration="4.820311562s" totalImagesPullingTime="2.822204955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:52.818460084 +0000 UTC m=+162.255136863" watchObservedRunningTime="2026-09-17 15:56:52.820311562 +0000 UTC m=+162.256988345"
Sep 17 15:56:53.795069 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:53.795022    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr" event={"ID":"8cbca560-0e32-447b-a0ee-5c2280afca4a","Type":"ContainerStarted","Data":"2826bed754b44c6e34deaef954303e7886a1d71dd77493d3c39fdff5e6bd39c2"}
Sep 17 15:56:53.795504 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:53.795419    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:53.795504 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:53.795456    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:53.800686 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:53.800661    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr"
Sep 17 15:56:53.819310 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:53.819258    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-f58fr" podStartSLOduration=3.19152047 podStartE2EDuration="4.81924278s" totalImagesPullingTime="1.62772231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:53.818973793 +0000 UTC m=+163.255650574" watchObservedRunningTime="2026-09-17 15:56:53.81924278 +0000 UTC m=+163.255919563"
Sep 17 15:56:54.030972 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:54.030939    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:56:54.033755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:54.033722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0be3ee88-c394-4754-8569-fde1369da3e1-metrics-tls\") pod \"dns-default-6qjsh\" (UID: \"0be3ee88-c394-4754-8569-fde1369da3e1\") " pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:56:54.248119 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:54.248088    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nhj85\""
Sep 17 15:56:54.256251 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:54.256221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:56:55.805507 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:55.805463    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb8f454b8-rj8db" event={"ID":"ee215151-aa4d-4df0-813a-61f9eef4e964","Type":"ContainerStarted","Data":"0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431"}
Sep 17 15:56:55.830826 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:55.830740    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fb8f454b8-rj8db" podStartSLOduration=1.802641206 podStartE2EDuration="5.830726086s" totalImagesPullingTime="4.02808488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:55.828436833 +0000 UTC m=+165.265113613" watchObservedRunningTime="2026-09-17 15:56:55.830726086 +0000 UTC m=+165.267402865"
Sep 17 15:56:55.851368 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:55.851287    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6qjsh"]
Sep 17 15:56:55.853919 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:55.853877    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be3ee88_c394_4754_8569_fde1369da3e1.slice/crio-d6b289146649050012dddf9cbf0b26bf331cd4aa764cfb0df0dac656392485e6 WatchSource:0}: Error finding container d6b289146649050012dddf9cbf0b26bf331cd4aa764cfb0df0dac656392485e6: Status 404 returned error can't find the container with id d6b289146649050012dddf9cbf0b26bf331cd4aa764cfb0df0dac656392485e6
Sep 17 15:56:56.811829 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:56.811785    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6qjsh" event={"ID":"0be3ee88-c394-4754-8569-fde1369da3e1","Type":"ContainerStarted","Data":"d6b289146649050012dddf9cbf0b26bf331cd4aa764cfb0df0dac656392485e6"}
Sep 17 15:56:57.817760 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:57.817683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6qjsh" event={"ID":"0be3ee88-c394-4754-8569-fde1369da3e1","Type":"ContainerStarted","Data":"dde1159e6abc13a75999248674342acc2112db08f62ec5e3ff8ebd993f9b6990"}
Sep 17 15:56:57.817760 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:57.817738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6qjsh" event={"ID":"0be3ee88-c394-4754-8569-fde1369da3e1","Type":"ContainerStarted","Data":"01843961ddd2c628425ba42639f1f6da07af1dcf99fa0fb17dafc5f5a0730416"}
Sep 17 15:56:57.818092 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:57.817770    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:56:57.840073 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:57.840030    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6qjsh" podStartSLOduration=131.177034711 podStartE2EDuration="2m12.84001499s" totalImagesPullingTime="1.662980279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:57.838606012 +0000 UTC m=+167.275282791" watchObservedRunningTime="2026-09-17 15:56:57.84001499 +0000 UTC m=+167.276691768"
Sep 17 15:56:58.419676 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.419640    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5ccddcc6f5-p6x2m"]
Sep 17 15:56:58.422930 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.422915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.431192 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.431151    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 15:56:58.435112 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.435088    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ccddcc6f5-p6x2m"]
Sep 17 15:56:58.466472 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.466437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-oauth-config\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.466586 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.466488    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-serving-cert\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.466586 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.466550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t9p5\" (UniqueName: \"kubernetes.io/projected/b02db07d-c5fa-4eaf-8260-6468e607e19f-kube-api-access-2t9p5\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.466650 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.466618    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-config\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.466698 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.466673    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-trusted-ca-bundle\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.466728 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.466719    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-oauth-serving-cert\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.466769 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.466753    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-service-ca\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.567782 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.567758    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t9p5\" (UniqueName: \"kubernetes.io/projected/b02db07d-c5fa-4eaf-8260-6468e607e19f-kube-api-access-2t9p5\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.567899 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.567791    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-config\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.567899 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.567835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-trusted-ca-bundle\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.567899 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.567853    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-oauth-serving-cert\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.567899 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.567889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-service-ca\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.568091 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.568023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-oauth-config\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.568141 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.568114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-serving-cert\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.568734 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.568704    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-config\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.568851 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.568833    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-service-ca\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.568955 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.568931    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-oauth-serving-cert\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.569330 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.569310    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-trusted-ca-bundle\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.570568 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.570547    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-oauth-config\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.570638 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.570617    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-serving-cert\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.576353 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.576327    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t9p5\" (UniqueName: \"kubernetes.io/projected/b02db07d-c5fa-4eaf-8260-6468e607e19f-kube-api-access-2t9p5\") pod \"console-5ccddcc6f5-p6x2m\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") " pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:58.732234 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:58.732129    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:56:59.009058 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:59.009019    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ccddcc6f5-p6x2m"]
Sep 17 15:56:59.011243 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:56:59.011212    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02db07d_c5fa_4eaf_8260_6468e607e19f.slice/crio-e0c3d70cbb94ecb7b6030ebdc0451f7021a7d5b3277ece9e3077bfa8acffb1b8 WatchSource:0}: Error finding container e0c3d70cbb94ecb7b6030ebdc0451f7021a7d5b3277ece9e3077bfa8acffb1b8: Status 404 returned error can't find the container with id e0c3d70cbb94ecb7b6030ebdc0451f7021a7d5b3277ece9e3077bfa8acffb1b8
Sep 17 15:56:59.690966 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:59.690937    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c6965dbd6-kpr7x"
Sep 17 15:56:59.805278 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:59.805254    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-9f5cb6d45-scmg9"
Sep 17 15:56:59.827399 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:59.827349    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccddcc6f5-p6x2m" event={"ID":"b02db07d-c5fa-4eaf-8260-6468e607e19f","Type":"ContainerStarted","Data":"1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513"}
Sep 17 15:56:59.827563 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:59.827412    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccddcc6f5-p6x2m" event={"ID":"b02db07d-c5fa-4eaf-8260-6468e607e19f","Type":"ContainerStarted","Data":"e0c3d70cbb94ecb7b6030ebdc0451f7021a7d5b3277ece9e3077bfa8acffb1b8"}
Sep 17 15:56:59.850757 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:56:59.850714    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5ccddcc6f5-p6x2m" podStartSLOduration=1.850699354 podStartE2EDuration="1.850699354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:59.849237537 +0000 UTC m=+169.285914316" watchObservedRunningTime="2026-09-17 15:56:59.850699354 +0000 UTC m=+169.287376132"
Sep 17 15:57:01.270089 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:01.270050    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:57:01.270481 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:01.270105    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:57:01.274802 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:01.274782    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:57:01.839020 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:01.838996    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:57:02.026846 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.026808    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" podUID="2ab36b25-1767-452f-8b74-520b39589954" containerName="registry" containerID="cri-o://21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd" gracePeriod=30
Sep 17 15:57:02.265639 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.265617    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:57:02.299938 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.299903    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2ab36b25-1767-452f-8b74-520b39589954-ca-trust-extracted\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.299991    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-registry-certificates\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300028    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-bound-sa-token\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300072    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300103    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-image-registry-private-configuration\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300142    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b24q\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-kube-api-access-4b24q\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300193    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-trusted-ca\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300327 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300223    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-installation-pull-secrets\") pod \"2ab36b25-1767-452f-8b74-520b39589954\" (UID: \"2ab36b25-1767-452f-8b74-520b39589954\") "
Sep 17 15:57:02.300658 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300400    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:57:02.300658 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.300505    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-registry-certificates\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.301034 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.301005    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:57:02.302831 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.302798    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:57:02.302923 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.302826    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:57:02.303076 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.303047    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:57:02.303313 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.303285    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:57:02.303393 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.303358    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-kube-api-access-4b24q" (OuterVolumeSpecName: "kube-api-access-4b24q") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "kube-api-access-4b24q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:57:02.308960 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.308932    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab36b25-1767-452f-8b74-520b39589954-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2ab36b25-1767-452f-8b74-520b39589954" (UID: "2ab36b25-1767-452f-8b74-520b39589954"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 15:57:02.401562 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.401483    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-image-registry-private-configuration\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.401562 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.401517    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b24q\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-kube-api-access-4b24q\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.401562 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.401530    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2ab36b25-1767-452f-8b74-520b39589954-trusted-ca\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.401562 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.401543    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2ab36b25-1767-452f-8b74-520b39589954-installation-pull-secrets\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.401562 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.401555    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2ab36b25-1767-452f-8b74-520b39589954-ca-trust-extracted\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.401775 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.401567    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-bound-sa-token\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.401775 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.401579    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2ab36b25-1767-452f-8b74-520b39589954-registry-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:02.840410 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.840374    2566 generic.go:356] "Generic (PLEG): container finished" podID="2ab36b25-1767-452f-8b74-520b39589954" containerID="21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd" exitCode=0
Sep 17 15:57:02.840545 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.840431    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p"
Sep 17 15:57:02.840545 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.840468    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" event={"ID":"2ab36b25-1767-452f-8b74-520b39589954","Type":"ContainerDied","Data":"21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd"}
Sep 17 15:57:02.840612 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.840546    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66cf577dcf-2wh9p" event={"ID":"2ab36b25-1767-452f-8b74-520b39589954","Type":"ContainerDied","Data":"ea5687f02da5c10f515891b71d7b2f35296b0119f29f692e8d955ca37819842c"}
Sep 17 15:57:02.840612 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.840569    2566 scope.go:117] "RemoveContainer" containerID="21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd"
Sep 17 15:57:02.850436 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.850413    2566 scope.go:117] "RemoveContainer" containerID="21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd"
Sep 17 15:57:02.850688 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:57:02.850669    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd\": container with ID starting with 21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd not found: ID does not exist" containerID="21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd"
Sep 17 15:57:02.850735 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.850695    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd"} err="failed to get container status \"21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd\": rpc error: code = NotFound desc = could not find container \"21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd\": container with ID starting with 21eec06ee213a9c8613b55295675a3fa25cfcb28ce7a6ef49474b56bdae493dd not found: ID does not exist"
Sep 17 15:57:02.862763 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.862738    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66cf577dcf-2wh9p"]
Sep 17 15:57:02.866655 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:02.866633    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66cf577dcf-2wh9p"]
Sep 17 15:57:03.044571 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:03.044542    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab36b25-1767-452f-8b74-520b39589954" path="/var/lib/kubelet/pods/2ab36b25-1767-452f-8b74-520b39589954/volumes"
Sep 17 15:57:07.823415 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:07.823338    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6qjsh"
Sep 17 15:57:08.732658 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:08.732621    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:57:08.732852 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:08.732677    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:57:08.737523 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:08.737484    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:57:08.868892 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:08.868860    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:57:08.920660 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:08.920631    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb8f454b8-rj8db"]
Sep 17 15:57:09.871226 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:09.871195    2566 generic.go:356] "Generic (PLEG): container finished" podID="f02766b9-9dda-442e-8486-b9865cae794f" containerID="156990274c9a020b36258153015208b99b73f7dc4124a787f58007e9369bcbe0" exitCode=0
Sep 17 15:57:09.871548 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:09.871269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n" event={"ID":"f02766b9-9dda-442e-8486-b9865cae794f","Type":"ContainerDied","Data":"156990274c9a020b36258153015208b99b73f7dc4124a787f58007e9369bcbe0"}
Sep 17 15:57:09.871695 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:09.871679    2566 scope.go:117] "RemoveContainer" containerID="156990274c9a020b36258153015208b99b73f7dc4124a787f58007e9369bcbe0"
Sep 17 15:57:10.877280 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:10.877246    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-7564n" event={"ID":"f02766b9-9dda-442e-8486-b9865cae794f","Type":"ContainerStarted","Data":"930c8efd78039c8fc4f4ec0380cbdcc8b29917aaeabfd6cf3c20ff6c62819534"}
Sep 17 15:57:14.893253 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:14.893223    2566 generic.go:356] "Generic (PLEG): container finished" podID="ccb002d7-89bf-4dd2-8909-8701ab7755f6" containerID="3b1db79201353dc8b49a2c7b71482d2893a819dc395322acaeb10813becdad41" exitCode=0
Sep 17 15:57:14.893572 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:14.893284    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-btkfl" event={"ID":"ccb002d7-89bf-4dd2-8909-8701ab7755f6","Type":"ContainerDied","Data":"3b1db79201353dc8b49a2c7b71482d2893a819dc395322acaeb10813becdad41"}
Sep 17 15:57:14.893626 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:14.893611    2566 scope.go:117] "RemoveContainer" containerID="3b1db79201353dc8b49a2c7b71482d2893a819dc395322acaeb10813becdad41"
Sep 17 15:57:15.899500 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:15.899462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-btkfl" event={"ID":"ccb002d7-89bf-4dd2-8909-8701ab7755f6","Type":"ContainerStarted","Data":"8522c8ed6870f90dace0501678782e69d830f4c974791712b877d218af84c74d"}
Sep 17 15:57:33.945885 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:33.945849    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fb8f454b8-rj8db" podUID="ee215151-aa4d-4df0-813a-61f9eef4e964" containerName="console" containerID="cri-o://0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431" gracePeriod=15
Sep 17 15:57:34.188066 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.188043    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:57:34.365523 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.365484    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-oauth-serving-cert\") pod \"ee215151-aa4d-4df0-813a-61f9eef4e964\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") "
Sep 17 15:57:34.365693 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.365529    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-service-ca\") pod \"ee215151-aa4d-4df0-813a-61f9eef4e964\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") "
Sep 17 15:57:34.365693 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.365568    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-oauth-config\") pod \"ee215151-aa4d-4df0-813a-61f9eef4e964\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") "
Sep 17 15:57:34.365693 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.365594    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhkbh\" (UniqueName: \"kubernetes.io/projected/ee215151-aa4d-4df0-813a-61f9eef4e964-kube-api-access-bhkbh\") pod \"ee215151-aa4d-4df0-813a-61f9eef4e964\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") "
Sep 17 15:57:34.365693 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.365626    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-console-config\") pod \"ee215151-aa4d-4df0-813a-61f9eef4e964\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") "
Sep 17 15:57:34.365693 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.365657    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-serving-cert\") pod \"ee215151-aa4d-4df0-813a-61f9eef4e964\" (UID: \"ee215151-aa4d-4df0-813a-61f9eef4e964\") "
Sep 17 15:57:34.366055 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.366019    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ee215151-aa4d-4df0-813a-61f9eef4e964" (UID: "ee215151-aa4d-4df0-813a-61f9eef4e964"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:57:34.366205 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.366078    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-service-ca" (OuterVolumeSpecName: "service-ca") pod "ee215151-aa4d-4df0-813a-61f9eef4e964" (UID: "ee215151-aa4d-4df0-813a-61f9eef4e964"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:57:34.366205 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.366095    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-console-config" (OuterVolumeSpecName: "console-config") pod "ee215151-aa4d-4df0-813a-61f9eef4e964" (UID: "ee215151-aa4d-4df0-813a-61f9eef4e964"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:57:34.367977 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.367953    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ee215151-aa4d-4df0-813a-61f9eef4e964" (UID: "ee215151-aa4d-4df0-813a-61f9eef4e964"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:57:34.368035 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.368016    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ee215151-aa4d-4df0-813a-61f9eef4e964" (UID: "ee215151-aa4d-4df0-813a-61f9eef4e964"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:57:34.368100 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.368077    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee215151-aa4d-4df0-813a-61f9eef4e964-kube-api-access-bhkbh" (OuterVolumeSpecName: "kube-api-access-bhkbh") pod "ee215151-aa4d-4df0-813a-61f9eef4e964" (UID: "ee215151-aa4d-4df0-813a-61f9eef4e964"). InnerVolumeSpecName "kube-api-access-bhkbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:57:34.466986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.466954    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-oauth-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:34.466986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.466980    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-service-ca\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:34.466986 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.466989    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-oauth-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:34.467246 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.466997    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhkbh\" (UniqueName: \"kubernetes.io/projected/ee215151-aa4d-4df0-813a-61f9eef4e964-kube-api-access-bhkbh\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:34.467246 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.467006    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee215151-aa4d-4df0-813a-61f9eef4e964-console-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:34.467246 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.467014    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee215151-aa4d-4df0-813a-61f9eef4e964-console-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:57:34.982524 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.982489    2566 generic.go:356] "Generic (PLEG): container finished" podID="ee215151-aa4d-4df0-813a-61f9eef4e964" containerID="0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431" exitCode=0
Sep 17 15:57:34.982948 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.982531    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb8f454b8-rj8db"
Sep 17 15:57:34.982948 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.982555    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb8f454b8-rj8db" event={"ID":"ee215151-aa4d-4df0-813a-61f9eef4e964","Type":"ContainerDied","Data":"0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431"}
Sep 17 15:57:34.982948 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.982620    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb8f454b8-rj8db" event={"ID":"ee215151-aa4d-4df0-813a-61f9eef4e964","Type":"ContainerDied","Data":"bab9b5163dac6003ff0a244c318aea1c81242341874ba6c0ca3d4345b25b5cdd"}
Sep 17 15:57:34.982948 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.982640    2566 scope.go:117] "RemoveContainer" containerID="0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431"
Sep 17 15:57:34.991016 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.991000    2566 scope.go:117] "RemoveContainer" containerID="0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431"
Sep 17 15:57:34.991278 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:57:34.991262    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431\": container with ID starting with 0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431 not found: ID does not exist" containerID="0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431"
Sep 17 15:57:34.991335 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:34.991291    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431"} err="failed to get container status \"0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431\": rpc error: code = NotFound desc = could not find container \"0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431\": container with ID starting with 0fc88805d62cb0c50071d478302e4dbaa31f2ca921e110a23d165c00a5277431 not found: ID does not exist"
Sep 17 15:57:35.003889 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:35.003832    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb8f454b8-rj8db"]
Sep 17 15:57:35.005472 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:35.005454    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fb8f454b8-rj8db"]
Sep 17 15:57:35.044179 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:57:35.044134    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee215151-aa4d-4df0-813a-61f9eef4e964" path="/var/lib/kubelet/pods/ee215151-aa4d-4df0-813a-61f9eef4e964/volumes"
Sep 17 15:58:05.528724 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:05.528689    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:58:05.529208 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:05.528967    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="alertmanager" containerID="cri-o://2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f" gracePeriod=120
Sep 17 15:58:05.529208 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:05.529031    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-metric" containerID="cri-o://d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3" gracePeriod=120
Sep 17 15:58:05.529208 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:05.529043    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-web" containerID="cri-o://10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0" gracePeriod=120
Sep 17 15:58:05.529208 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:05.529094    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy" containerID="cri-o://f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880" gracePeriod=120
Sep 17 15:58:05.529208 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:05.529120    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="config-reloader" containerID="cri-o://95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b" gracePeriod=120
Sep 17 15:58:05.529493 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:05.529116    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="prom-label-proxy" containerID="cri-o://443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136" gracePeriod=120
Sep 17 15:58:06.108998 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.108960    2566 generic.go:356] "Generic (PLEG): container finished" podID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerID="443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136" exitCode=0
Sep 17 15:58:06.108998 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.108989    2566 generic.go:356] "Generic (PLEG): container finished" podID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerID="f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880" exitCode=0
Sep 17 15:58:06.108998 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.108999    2566 generic.go:356] "Generic (PLEG): container finished" podID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerID="95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b" exitCode=0
Sep 17 15:58:06.108998 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.109007    2566 generic.go:356] "Generic (PLEG): container finished" podID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerID="2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f" exitCode=0
Sep 17 15:58:06.109287 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.109026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"}
Sep 17 15:58:06.109287 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.109089    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"}
Sep 17 15:58:06.109287 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.109116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"}
Sep 17 15:58:06.109287 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.109140    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"}
Sep 17 15:58:06.765269 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.765245    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:06.784723 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784701    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-cluster-tls-config\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.784821 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784742    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-out\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.784821 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784772    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nd5k\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-kube-api-access-8nd5k\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.784821 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784805    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-metrics-client-ca\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.784969 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784837    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.784969 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784875    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-trusted-ca-bundle\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.784969 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784917    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-web\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.784969 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784949    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.785185 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.784989    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-main-db\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.785185 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785031    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.785185 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785077    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-tls-assets\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.785185 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785110    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-volume\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.785185 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785135    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-web-config\") pod \"da4b96de-fe9d-4eef-8384-ebe5b996be26\" (UID: \"da4b96de-fe9d-4eef-8384-ebe5b996be26\") "
Sep 17 15:58:06.785427 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785232    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:58:06.785427 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785410    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 15:58:06.785539 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785512    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-main-db\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.785539 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785532    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-metrics-client-ca\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.786240 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.785967    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:58:06.787903 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.787873    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:06.788021 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.787995    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-out" (OuterVolumeSpecName: "config-out") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 15:58:06.788742 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.788714    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:06.788897 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.788779    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-kube-api-access-8nd5k" (OuterVolumeSpecName: "kube-api-access-8nd5k") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "kube-api-access-8nd5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:58:06.789052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.789018    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:06.789290 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.789270    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-volume" (OuterVolumeSpecName: "config-volume") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:06.789565 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.789544    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:58:06.789853 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.789836    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:06.792636 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.792619    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:06.800344 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.800319    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-web-config" (OuterVolumeSpecName: "web-config") pod "da4b96de-fe9d-4eef-8384-ebe5b996be26" (UID: "da4b96de-fe9d-4eef-8384-ebe5b996be26"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:06.886908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886842    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-main-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.886908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886869    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-tls-assets\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.886908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886878    2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-volume\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.886908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886886    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-web-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.886908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886893    2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-cluster-tls-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.886908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886902    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da4b96de-fe9d-4eef-8384-ebe5b996be26-config-out\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.886908 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886910    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nd5k\" (UniqueName: \"kubernetes.io/projected/da4b96de-fe9d-4eef-8384-ebe5b996be26-kube-api-access-8nd5k\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.887221 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886919    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.887221 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886930    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4b96de-fe9d-4eef-8384-ebe5b996be26-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.887221 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886941    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:06.887221 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:06.886950    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da4b96de-fe9d-4eef-8384-ebe5b996be26-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:07.116043 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.116014    2566 generic.go:356] "Generic (PLEG): container finished" podID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerID="d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3" exitCode=0
Sep 17 15:58:07.116043 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.116039    2566 generic.go:356] "Generic (PLEG): container finished" podID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerID="10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0" exitCode=0
Sep 17 15:58:07.116218 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.116062    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"}
Sep 17 15:58:07.116218 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.116098    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.116218 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.116144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"}
Sep 17 15:58:07.116218 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.116179    2566 scope.go:117] "RemoveContainer" containerID="443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"
Sep 17 15:58:07.116218 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.116209    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da4b96de-fe9d-4eef-8384-ebe5b996be26","Type":"ContainerDied","Data":"141dc05b164c8678b9beeae6f0637911663c5a647f0088ea645b79a3ba870862"}
Sep 17 15:58:07.123654 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.123639    2566 scope.go:117] "RemoveContainer" containerID="d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"
Sep 17 15:58:07.130353 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.130322    2566 scope.go:117] "RemoveContainer" containerID="f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"
Sep 17 15:58:07.137621 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.137594    2566 scope.go:117] "RemoveContainer" containerID="10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"
Sep 17 15:58:07.137806 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.137786    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:58:07.140939 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.140918    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:58:07.144421 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.144404    2566 scope.go:117] "RemoveContainer" containerID="95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"
Sep 17 15:58:07.151128 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.151113    2566 scope.go:117] "RemoveContainer" containerID="2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"
Sep 17 15:58:07.158008 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.157989    2566 scope.go:117] "RemoveContainer" containerID="1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c"
Sep 17 15:58:07.164272 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.164256    2566 scope.go:117] "RemoveContainer" containerID="443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"
Sep 17 15:58:07.164509 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:07.164491    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136\": container with ID starting with 443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136 not found: ID does not exist" containerID="443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"
Sep 17 15:58:07.164563 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.164517    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"} err="failed to get container status \"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136\": rpc error: code = NotFound desc = could not find container \"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136\": container with ID starting with 443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136 not found: ID does not exist"
Sep 17 15:58:07.164563 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.164535    2566 scope.go:117] "RemoveContainer" containerID="d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"
Sep 17 15:58:07.164796 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:07.164777    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3\": container with ID starting with d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3 not found: ID does not exist" containerID="d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"
Sep 17 15:58:07.164842 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.164802    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"} err="failed to get container status \"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3\": rpc error: code = NotFound desc = could not find container \"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3\": container with ID starting with d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3 not found: ID does not exist"
Sep 17 15:58:07.164842 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.164817    2566 scope.go:117] "RemoveContainer" containerID="f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"
Sep 17 15:58:07.165011 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:07.164995    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880\": container with ID starting with f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880 not found: ID does not exist" containerID="f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"
Sep 17 15:58:07.165051 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165016    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"} err="failed to get container status \"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880\": rpc error: code = NotFound desc = could not find container \"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880\": container with ID starting with f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880 not found: ID does not exist"
Sep 17 15:58:07.165051 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165029    2566 scope.go:117] "RemoveContainer" containerID="10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"
Sep 17 15:58:07.165284 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:07.165266    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0\": container with ID starting with 10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0 not found: ID does not exist" containerID="10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"
Sep 17 15:58:07.165359 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165294    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"} err="failed to get container status \"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0\": rpc error: code = NotFound desc = could not find container \"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0\": container with ID starting with 10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0 not found: ID does not exist"
Sep 17 15:58:07.165359 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165313    2566 scope.go:117] "RemoveContainer" containerID="95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"
Sep 17 15:58:07.165542 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:07.165526    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b\": container with ID starting with 95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b not found: ID does not exist" containerID="95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"
Sep 17 15:58:07.165584 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165546    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"} err="failed to get container status \"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b\": rpc error: code = NotFound desc = could not find container \"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b\": container with ID starting with 95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b not found: ID does not exist"
Sep 17 15:58:07.165584 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165560    2566 scope.go:117] "RemoveContainer" containerID="2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"
Sep 17 15:58:07.165819 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:07.165803    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f\": container with ID starting with 2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f not found: ID does not exist" containerID="2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"
Sep 17 15:58:07.165868 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165820    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"} err="failed to get container status \"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f\": rpc error: code = NotFound desc = could not find container \"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f\": container with ID starting with 2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f not found: ID does not exist"
Sep 17 15:58:07.165868 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.165831    2566 scope.go:117] "RemoveContainer" containerID="1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c"
Sep 17 15:58:07.166038 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:07.166020    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c\": container with ID starting with 1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c not found: ID does not exist" containerID="1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c"
Sep 17 15:58:07.166077 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166043    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c"} err="failed to get container status \"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c\": rpc error: code = NotFound desc = could not find container \"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c\": container with ID starting with 1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c not found: ID does not exist"
Sep 17 15:58:07.166077 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166057    2566 scope.go:117] "RemoveContainer" containerID="443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"
Sep 17 15:58:07.166333 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166314    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136"} err="failed to get container status \"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136\": rpc error: code = NotFound desc = could not find container \"443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136\": container with ID starting with 443f24feada884053df2627c651d8bc05d3d6c422c567f73ecf497989484e136 not found: ID does not exist"
Sep 17 15:58:07.166333 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166333    2566 scope.go:117] "RemoveContainer" containerID="d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"
Sep 17 15:58:07.166585 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166563    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3"} err="failed to get container status \"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3\": rpc error: code = NotFound desc = could not find container \"d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3\": container with ID starting with d04c8983fb759bdb2de6732c3756a699d5efeb2492cdf393e83948c2c0f097c3 not found: ID does not exist"
Sep 17 15:58:07.166585 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166584    2566 scope.go:117] "RemoveContainer" containerID="f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"
Sep 17 15:58:07.166842 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166826    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880"} err="failed to get container status \"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880\": rpc error: code = NotFound desc = could not find container \"f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880\": container with ID starting with f7bd2a5c1d004044662b3db09dd120d4df6bc6f5d2643a62a2086475b51b0880 not found: ID does not exist"
Sep 17 15:58:07.166842 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.166841    2566 scope.go:117] "RemoveContainer" containerID="10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"
Sep 17 15:58:07.167071 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.167054    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0"} err="failed to get container status \"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0\": rpc error: code = NotFound desc = could not find container \"10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0\": container with ID starting with 10a7b673cca1c172cd6aaf2aba5660e950a9d5d2a3f63daa3f15230df04ae5d0 not found: ID does not exist"
Sep 17 15:58:07.167071 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.167071    2566 scope.go:117] "RemoveContainer" containerID="95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"
Sep 17 15:58:07.167311 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.167293    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b"} err="failed to get container status \"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b\": rpc error: code = NotFound desc = could not find container \"95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b\": container with ID starting with 95d1f4d3122849e2e9d33d394d904b64e0c363e06a006a9bb910befee4e9633b not found: ID does not exist"
Sep 17 15:58:07.167363 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.167312    2566 scope.go:117] "RemoveContainer" containerID="2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"
Sep 17 15:58:07.167591 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.167569    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f"} err="failed to get container status \"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f\": rpc error: code = NotFound desc = could not find container \"2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f\": container with ID starting with 2c4aba93843b5ec2133f20417d6e5ac68df5b9a2c4d430d4f86c1469253b0f4f not found: ID does not exist"
Sep 17 15:58:07.167591 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.167590    2566 scope.go:117] "RemoveContainer" containerID="1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c"
Sep 17 15:58:07.167829 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.167812    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c"} err="failed to get container status \"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c\": rpc error: code = NotFound desc = could not find container \"1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c\": container with ID starting with 1d1b4532dfcea0f9799952dc524eec9d76ae4b1030cd1d3224a5ccef4114591c not found: ID does not exist"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.183978    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184605    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-metric"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184623    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-metric"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184651    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-web"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184660    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-web"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184679    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184689    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184700    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="prom-label-proxy"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184710    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="prom-label-proxy"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184731    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="config-reloader"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184739    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="config-reloader"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184756    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="alertmanager"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184766    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="alertmanager"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184779    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab36b25-1767-452f-8b74-520b39589954" containerName="registry"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184787    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab36b25-1767-452f-8b74-520b39589954" containerName="registry"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184797    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="init-config-reloader"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184806    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="init-config-reloader"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184834    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee215151-aa4d-4df0-813a-61f9eef4e964" containerName="console"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184842    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee215151-aa4d-4df0-813a-61f9eef4e964" containerName="console"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184976    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.184990    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="alertmanager"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.185005    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="config-reloader"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.185013    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-metric"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.185024    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="kube-rbac-proxy-web"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.185032    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab36b25-1767-452f-8b74-520b39589954" containerName="registry"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.185048    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee215151-aa4d-4df0-813a-61f9eef4e964" containerName="console"
Sep 17 15:58:07.187187 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.185057    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" containerName="prom-label-proxy"
Sep 17 15:58:07.193087 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.193066    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.195722 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.195701    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 15:58:07.195820 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.195725    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 15:58:07.195883 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.195833    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4c7sb\""
Sep 17 15:58:07.195883 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.195852    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 15:58:07.196324 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.196305    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 15:58:07.196428 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.196312    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 15:58:07.196428 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.196358    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 15:58:07.196428 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.196325    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 15:58:07.196589 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.196330    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 15:58:07.202172 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.202139    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:58:07.202415 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.202389    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 15:58:07.289485 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289601 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289497    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1cf02ae-3902-4438-9b81-6c950cd5f195-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289601 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289517    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289601 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289534    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-web-config\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289601 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1cf02ae-3902-4438-9b81-6c950cd5f195-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289736 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289654    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289736 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289688    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289736 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4n9t\" (UniqueName: \"kubernetes.io/projected/f1cf02ae-3902-4438-9b81-6c950cd5f195-kube-api-access-l4n9t\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289736 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289725    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1cf02ae-3902-4438-9b81-6c950cd5f195-config-out\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289853 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289741    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289853 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1cf02ae-3902-4438-9b81-6c950cd5f195-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289853 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.289853 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.289796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1cf02ae-3902-4438-9b81-6c950cd5f195-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390094 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1cf02ae-3902-4438-9b81-6c950cd5f195-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390094 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390094 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390084    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-web-config\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390314 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1cf02ae-3902-4438-9b81-6c950cd5f195-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390314 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390236    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390314 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390284    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390464 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390318    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4n9t\" (UniqueName: \"kubernetes.io/projected/f1cf02ae-3902-4438-9b81-6c950cd5f195-kube-api-access-l4n9t\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390464 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390345    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1cf02ae-3902-4438-9b81-6c950cd5f195-config-out\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390464 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390370    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390464 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390400    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1cf02ae-3902-4438-9b81-6c950cd5f195-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390464 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390428    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390464 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1cf02ae-3902-4438-9b81-6c950cd5f195-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390761 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390521    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.390834 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390811    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1cf02ae-3902-4438-9b81-6c950cd5f195-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.391236 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.390893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1cf02ae-3902-4438-9b81-6c950cd5f195-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393194 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393152    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1cf02ae-3902-4438-9b81-6c950cd5f195-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393297 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393219    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393376 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393353    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-web-config\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393499 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393481    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393550 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393484    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393593 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393564    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393632 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393589    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393860 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393837    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1cf02ae-3902-4438-9b81-6c950cd5f195-config-out\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.393939 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.393839    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1cf02ae-3902-4438-9b81-6c950cd5f195-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.395074 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.395058    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1cf02ae-3902-4438-9b81-6c950cd5f195-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.399577 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.399556    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4n9t\" (UniqueName: \"kubernetes.io/projected/f1cf02ae-3902-4438-9b81-6c950cd5f195-kube-api-access-l4n9t\") pod \"alertmanager-main-0\" (UID: \"f1cf02ae-3902-4438-9b81-6c950cd5f195\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.503819 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.503791    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 15:58:07.789320 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:07.789295    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 15:58:07.791236 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:58:07.791212    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1cf02ae_3902_4438_9b81_6c950cd5f195.slice/crio-159164386b3191bcdf3338be1810173560a60c7d33a5593217968ec9648644ac WatchSource:0}: Error finding container 159164386b3191bcdf3338be1810173560a60c7d33a5593217968ec9648644ac: Status 404 returned error can't find the container with id 159164386b3191bcdf3338be1810173560a60c7d33a5593217968ec9648644ac
Sep 17 15:58:08.121557 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:08.121480    2566 generic.go:356] "Generic (PLEG): container finished" podID="f1cf02ae-3902-4438-9b81-6c950cd5f195" containerID="2e774df239194dfb76eb005ee233c8f88244dae6e6568133f53cc6d8011d2eed" exitCode=0
Sep 17 15:58:08.121687 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:08.121568    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerDied","Data":"2e774df239194dfb76eb005ee233c8f88244dae6e6568133f53cc6d8011d2eed"}
Sep 17 15:58:08.121687 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:08.121634    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerStarted","Data":"159164386b3191bcdf3338be1810173560a60c7d33a5593217968ec9648644ac"}
Sep 17 15:58:09.044664 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.044632    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da4b96de-fe9d-4eef-8384-ebe5b996be26" path="/var/lib/kubelet/pods/da4b96de-fe9d-4eef-8384-ebe5b996be26/volumes"
Sep 17 15:58:09.130133 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.130097    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerStarted","Data":"da177760a1cfda5701dfa31c3fe83572be147a950b030e4cd2e2cc758068ece7"}
Sep 17 15:58:09.130281 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.130196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerStarted","Data":"6cbf4e1be13fa7d5d6ff3daf9c30979f0473e3647ae5c4d0c3a4fe7f7301821a"}
Sep 17 15:58:09.130281 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.130247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerStarted","Data":"40ad66bd4084fe83bff4afa18ef48861c0c4f9c73144e252dfbc8054fb55d10d"}
Sep 17 15:58:09.130417 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.130283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerStarted","Data":"5104f624e8c7a3649bbebdbf249a1288f5de30e7ea9285ecc05c302b8b11dbe0"}
Sep 17 15:58:09.130417 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.130323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerStarted","Data":"f2dea5b708c958bab96a113648fafceb097711bca0461e6edcddf905f111f903"}
Sep 17 15:58:09.130417 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.130361    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1cf02ae-3902-4438-9b81-6c950cd5f195","Type":"ContainerStarted","Data":"91822152cf9dbaf55fa4416a170a8679b3de624a3fd67082ff5a508d09376055"}
Sep 17 15:58:09.161279 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.161231    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.145464985 podStartE2EDuration="2.161216086s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.751101ms" isStatefulPod=false podCreationTimestamp="2026-09-17 15:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:58:09.159555717 +0000 UTC m=+238.596232495" watchObservedRunningTime="2026-09-17 15:58:09.161216086 +0000 UTC m=+238.597892864"
Sep 17 15:58:09.544375 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.544344    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-58cc499c6f-2tknt"]
Sep 17 15:58:09.548953 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.548934    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.551882 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.551858    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 17 15:58:09.551981 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.551959    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 17 15:58:09.552237 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.552222    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pw5qh\""
Sep 17 15:58:09.552331 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.552257    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 17 15:58:09.552412 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.552395    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 17 15:58:09.552540 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.552394    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 17 15:58:09.557974 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.557957    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 17 15:58:09.561950 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.561931    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58cc499c6f-2tknt"]
Sep 17 15:58:09.608653 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608629    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-federate-client-tls\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.608755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-metrics-client-ca\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.608755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608676    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.608755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608699    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.608755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-secret-telemeter-client\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.608917 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608800    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czxsz\" (UniqueName: \"kubernetes.io/projected/a30c2931-425e-4d1c-bfb8-f303d0b20faf-kube-api-access-czxsz\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.608917 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608839    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-serving-certs-ca-bundle\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.609019 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.608918    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-telemeter-client-tls\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.709888 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.709859    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710032 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.709899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710032 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.709924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-secret-telemeter-client\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710032 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.709944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czxsz\" (UniqueName: \"kubernetes.io/projected/a30c2931-425e-4d1c-bfb8-f303d0b20faf-kube-api-access-czxsz\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710032 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.709969    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-serving-certs-ca-bundle\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710032 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.710028    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-telemeter-client-tls\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710343 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.710077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-federate-client-tls\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710343 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.710097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-metrics-client-ca\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710842 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.710818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710928 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.710839    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-metrics-client-ca\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.710928 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.710821    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a30c2931-425e-4d1c-bfb8-f303d0b20faf-serving-certs-ca-bundle\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.712575 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.712544    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.712662 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.712584    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-federate-client-tls\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.712787 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.712768    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-telemeter-client-tls\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.713018 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.712996    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a30c2931-425e-4d1c-bfb8-f303d0b20faf-secret-telemeter-client\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.718449 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.718433    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czxsz\" (UniqueName: \"kubernetes.io/projected/a30c2931-425e-4d1c-bfb8-f303d0b20faf-kube-api-access-czxsz\") pod \"telemeter-client-58cc499c6f-2tknt\" (UID: \"a30c2931-425e-4d1c-bfb8-f303d0b20faf\") " pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:09.859949 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:09.859893    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt"
Sep 17 15:58:10.139420 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:10.139334    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58cc499c6f-2tknt"]
Sep 17 15:58:10.143024 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:58:10.142986    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda30c2931_425e_4d1c_bfb8_f303d0b20faf.slice/crio-0b159337eae0d079b97b69cdd152d2b069538ea845d839d67cb960f53cc2e67b WatchSource:0}: Error finding container 0b159337eae0d079b97b69cdd152d2b069538ea845d839d67cb960f53cc2e67b: Status 404 returned error can't find the container with id 0b159337eae0d079b97b69cdd152d2b069538ea845d839d67cb960f53cc2e67b
Sep 17 15:58:11.139613 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:11.139574    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt" event={"ID":"a30c2931-425e-4d1c-bfb8-f303d0b20faf","Type":"ContainerStarted","Data":"0b159337eae0d079b97b69cdd152d2b069538ea845d839d67cb960f53cc2e67b"}
Sep 17 15:58:13.149833 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.149793    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt" event={"ID":"a30c2931-425e-4d1c-bfb8-f303d0b20faf","Type":"ContainerStarted","Data":"35be16363a36b00f7adfe86a54e6a1eea12bb22821d122e13f079b8b73ad8bc5"}
Sep 17 15:58:13.150201 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.149856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt" event={"ID":"a30c2931-425e-4d1c-bfb8-f303d0b20faf","Type":"ContainerStarted","Data":"b88bd4f776b512c3beea89adf5bd41684b66f6e57d67a91633ba99adbad57ab3"}
Sep 17 15:58:13.150201 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.149878    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt" event={"ID":"a30c2931-425e-4d1c-bfb8-f303d0b20faf","Type":"ContainerStarted","Data":"1b8ee39d3bd1339a48a88f35e2d7d7fe861788a20bfd997517e7243a89c15fa4"}
Sep 17 15:58:13.175373 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.175326    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-58cc499c6f-2tknt" podStartSLOduration=2.213097547 podStartE2EDuration="4.17531179s" totalImagesPullingTime="1.962214243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:58:13.172446194 +0000 UTC m=+242.609122973" watchObservedRunningTime="2026-09-17 15:58:13.17531179 +0000 UTC m=+242.611988568"
Sep 17 15:58:13.276149 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.276111    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7df77c8b88-j4wf7"]
Sep 17 15:58:13.279957 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.279936    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.291369 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.291345    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7df77c8b88-j4wf7"]
Sep 17 15:58:13.338900 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.338873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-service-ca\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.339003 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.338910    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-oauth-serving-cert\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.339038 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.339022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnhkf\" (UniqueName: \"kubernetes.io/projected/ab946318-0b50-45d2-b6cb-d6d340b73272-kube-api-access-xnhkf\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.339091 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.339078    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-console-config\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.339128 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.339102    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-trusted-ca-bundle\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.339174 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.339134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-oauth-config\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.339231 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.339176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-serving-cert\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440210 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440124    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-serving-cert\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440210 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440186    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-service-ca\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440373 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440221    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-oauth-serving-cert\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440409 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440368    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhkf\" (UniqueName: \"kubernetes.io/projected/ab946318-0b50-45d2-b6cb-d6d340b73272-kube-api-access-xnhkf\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440459 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-console-config\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440459 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-trusted-ca-bundle\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440555 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-oauth-config\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.440982 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.440942    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-oauth-serving-cert\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.441093 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.441067    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-console-config\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.441324 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.441299    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-service-ca\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.441390 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.441339    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-trusted-ca-bundle\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.442863 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.442840    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-oauth-config\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.442951 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.442840    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-serving-cert\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.450550 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.450528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnhkf\" (UniqueName: \"kubernetes.io/projected/ab946318-0b50-45d2-b6cb-d6d340b73272-kube-api-access-xnhkf\") pod \"console-7df77c8b88-j4wf7\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") " pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.591613 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.591578    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:13.871209 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:13.871180    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7df77c8b88-j4wf7"]
Sep 17 15:58:13.874486 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:58:13.874464    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab946318_0b50_45d2_b6cb_d6d340b73272.slice/crio-7da1abcb57231691b8c4636000f22d9438d93d160f8893e0b5d96d70ccbe9983 WatchSource:0}: Error finding container 7da1abcb57231691b8c4636000f22d9438d93d160f8893e0b5d96d70ccbe9983: Status 404 returned error can't find the container with id 7da1abcb57231691b8c4636000f22d9438d93d160f8893e0b5d96d70ccbe9983
Sep 17 15:58:14.155588 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:14.155504    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df77c8b88-j4wf7" event={"ID":"ab946318-0b50-45d2-b6cb-d6d340b73272","Type":"ContainerStarted","Data":"e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2"}
Sep 17 15:58:14.155588 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:14.155565    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df77c8b88-j4wf7" event={"ID":"ab946318-0b50-45d2-b6cb-d6d340b73272","Type":"ContainerStarted","Data":"7da1abcb57231691b8c4636000f22d9438d93d160f8893e0b5d96d70ccbe9983"}
Sep 17 15:58:14.174524 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:14.174475    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7df77c8b88-j4wf7" podStartSLOduration=1.174455165 podStartE2EDuration="1.174455165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:58:14.174258295 +0000 UTC m=+243.610935073" watchObservedRunningTime="2026-09-17 15:58:14.174455165 +0000 UTC m=+243.611131945"
Sep 17 15:58:23.592427 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:23.592391    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:23.592870 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:23.592455    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:23.597067 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:23.597048    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:24.200791 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:24.200761    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 15:58:24.251664 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:24.251629    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ccddcc6f5-p6x2m"]
Sep 17 15:58:49.271900 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.271865    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5ccddcc6f5-p6x2m" podUID="b02db07d-c5fa-4eaf-8260-6468e607e19f" containerName="console" containerID="cri-o://1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513" gracePeriod=15
Sep 17 15:58:49.508934 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.508910    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:58:49.616639 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.616570    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-config\") pod \"b02db07d-c5fa-4eaf-8260-6468e607e19f\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") "
Sep 17 15:58:49.616639 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.616622    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-oauth-serving-cert\") pod \"b02db07d-c5fa-4eaf-8260-6468e607e19f\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") "
Sep 17 15:58:49.616823 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.616670    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t9p5\" (UniqueName: \"kubernetes.io/projected/b02db07d-c5fa-4eaf-8260-6468e607e19f-kube-api-access-2t9p5\") pod \"b02db07d-c5fa-4eaf-8260-6468e607e19f\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") "
Sep 17 15:58:49.616823 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.616689    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-service-ca\") pod \"b02db07d-c5fa-4eaf-8260-6468e607e19f\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") "
Sep 17 15:58:49.616823 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.616731    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-serving-cert\") pod \"b02db07d-c5fa-4eaf-8260-6468e607e19f\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") "
Sep 17 15:58:49.616964 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.616844    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-oauth-config\") pod \"b02db07d-c5fa-4eaf-8260-6468e607e19f\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") "
Sep 17 15:58:49.616964 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.616886    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-trusted-ca-bundle\") pod \"b02db07d-c5fa-4eaf-8260-6468e607e19f\" (UID: \"b02db07d-c5fa-4eaf-8260-6468e607e19f\") "
Sep 17 15:58:49.617062 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.617014    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-config" (OuterVolumeSpecName: "console-config") pod "b02db07d-c5fa-4eaf-8260-6468e607e19f" (UID: "b02db07d-c5fa-4eaf-8260-6468e607e19f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:58:49.617138 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.617103    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b02db07d-c5fa-4eaf-8260-6468e607e19f" (UID: "b02db07d-c5fa-4eaf-8260-6468e607e19f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:58:49.617225 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.617106    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-service-ca" (OuterVolumeSpecName: "service-ca") pod "b02db07d-c5fa-4eaf-8260-6468e607e19f" (UID: "b02db07d-c5fa-4eaf-8260-6468e607e19f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:58:49.617377 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.617360    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-oauth-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:49.617450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.617380    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-service-ca\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:49.617450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.617390    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:49.617450 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.617402    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b02db07d-c5fa-4eaf-8260-6468e607e19f" (UID: "b02db07d-c5fa-4eaf-8260-6468e607e19f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 15:58:49.618810 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.618790    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b02db07d-c5fa-4eaf-8260-6468e607e19f-kube-api-access-2t9p5" (OuterVolumeSpecName: "kube-api-access-2t9p5") pod "b02db07d-c5fa-4eaf-8260-6468e607e19f" (UID: "b02db07d-c5fa-4eaf-8260-6468e607e19f"). InnerVolumeSpecName "kube-api-access-2t9p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 15:58:49.619292 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.619268    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b02db07d-c5fa-4eaf-8260-6468e607e19f" (UID: "b02db07d-c5fa-4eaf-8260-6468e607e19f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:49.619386 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.619286    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b02db07d-c5fa-4eaf-8260-6468e607e19f" (UID: "b02db07d-c5fa-4eaf-8260-6468e607e19f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 15:58:49.718111 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.718081    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t9p5\" (UniqueName: \"kubernetes.io/projected/b02db07d-c5fa-4eaf-8260-6468e607e19f-kube-api-access-2t9p5\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:49.718111 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.718104    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:49.718111 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.718114    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02db07d-c5fa-4eaf-8260-6468e607e19f-console-oauth-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:49.718324 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:49.718124    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02db07d-c5fa-4eaf-8260-6468e607e19f-trusted-ca-bundle\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 15:58:50.300290 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.300256    2566 generic.go:356] "Generic (PLEG): container finished" podID="b02db07d-c5fa-4eaf-8260-6468e607e19f" containerID="1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513" exitCode=0
Sep 17 15:58:50.300695 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.300299    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ccddcc6f5-p6x2m"
Sep 17 15:58:50.300695 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.300345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccddcc6f5-p6x2m" event={"ID":"b02db07d-c5fa-4eaf-8260-6468e607e19f","Type":"ContainerDied","Data":"1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513"}
Sep 17 15:58:50.300695 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.300393    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ccddcc6f5-p6x2m" event={"ID":"b02db07d-c5fa-4eaf-8260-6468e607e19f","Type":"ContainerDied","Data":"e0c3d70cbb94ecb7b6030ebdc0451f7021a7d5b3277ece9e3077bfa8acffb1b8"}
Sep 17 15:58:50.300695 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.300414    2566 scope.go:117] "RemoveContainer" containerID="1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513"
Sep 17 15:58:50.312490 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.312316    2566 scope.go:117] "RemoveContainer" containerID="1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513"
Sep 17 15:58:50.312781 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:50.312716    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513\": container with ID starting with 1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513 not found: ID does not exist" containerID="1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513"
Sep 17 15:58:50.312781 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.312752    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513"} err="failed to get container status \"1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513\": rpc error: code = NotFound desc = could not find container \"1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513\": container with ID starting with 1daf4088e160a640fa88a179567497f64eb9d4ba18ac028465eda875edae9513 not found: ID does not exist"
Sep 17 15:58:50.325791 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.325766    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ccddcc6f5-p6x2m"]
Sep 17 15:58:50.327389 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:50.327362    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02db07d_c5fa_4eaf_8260_6468e607e19f.slice/crio-e0c3d70cbb94ecb7b6030ebdc0451f7021a7d5b3277ece9e3077bfa8acffb1b8\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02db07d_c5fa_4eaf_8260_6468e607e19f.slice\": RecentStats: unable to find data in memory cache]"
Sep 17 15:58:50.327499 ip-10-0-131-144 kubenswrapper[2566]: E0917 15:58:50.327430    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02db07d_c5fa_4eaf_8260_6468e607e19f.slice/crio-e0c3d70cbb94ecb7b6030ebdc0451f7021a7d5b3277ece9e3077bfa8acffb1b8\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02db07d_c5fa_4eaf_8260_6468e607e19f.slice\": RecentStats: unable to find data in memory cache]"
Sep 17 15:58:50.333182 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:50.332722    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5ccddcc6f5-p6x2m"]
Sep 17 15:58:51.044336 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:58:51.044300    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b02db07d-c5fa-4eaf-8260-6468e607e19f" path="/var/lib/kubelet/pods/b02db07d-c5fa-4eaf-8260-6468e607e19f/volumes"
Sep 17 15:59:10.981052 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:10.981021    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 15:59:10.988696 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:10.988667    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 15:59:10.988946 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:10.988926    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 15:59:10.992493 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:10.992466    2566 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 15:59:10.997113 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:10.997094    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 15:59:38.431354 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.431318    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55bb4bf8d9-sthj9"]
Sep 17 15:59:38.433824 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.431815    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b02db07d-c5fa-4eaf-8260-6468e607e19f" containerName="console"
Sep 17 15:59:38.433824 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.431831    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b02db07d-c5fa-4eaf-8260-6468e607e19f" containerName="console"
Sep 17 15:59:38.433824 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.431897    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b02db07d-c5fa-4eaf-8260-6468e607e19f" containerName="console"
Sep 17 15:59:38.434762 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.434746    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.446755 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.446733    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55bb4bf8d9-sthj9"]
Sep 17 15:59:38.582563 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.582531    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-oauth-serving-cert\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.582563 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.582569    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-oauth-config\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.582760 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.582586    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv8d5\" (UniqueName: \"kubernetes.io/projected/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-kube-api-access-nv8d5\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.582760 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.582661    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-service-ca\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.582760 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.582692    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-trusted-ca-bundle\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.582879 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.582769    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-serving-cert\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.582879 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.582784    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-config\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684015 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.683918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-serving-cert\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684015 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.683963    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-config\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684015 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684002    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-oauth-serving-cert\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684347 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-oauth-config\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684347 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684058    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv8d5\" (UniqueName: \"kubernetes.io/projected/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-kube-api-access-nv8d5\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684347 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684096    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-service-ca\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684347 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684120    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-trusted-ca-bundle\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684849 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684826    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-oauth-serving-cert\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684950 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684866    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-service-ca\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.684950 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.684927    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-config\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.685095 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.685077    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-trusted-ca-bundle\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.686548 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.686529    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-oauth-config\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.686633 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.686589    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-serving-cert\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.692904 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.692878    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv8d5\" (UniqueName: \"kubernetes.io/projected/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-kube-api-access-nv8d5\") pod \"console-55bb4bf8d9-sthj9\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") " pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:38.744203 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:38.744182    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:39.015635 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:39.015610    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55bb4bf8d9-sthj9"]
Sep 17 15:59:39.017813 ip-10-0-131-144 kubenswrapper[2566]: W0917 15:59:39.017789    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a2da14_69c7_40fc_86ac_ce5b85ca836f.slice/crio-440aa14cdaa59d43e8c78054395cda560f1246789b89b8a96af68a3cc0485776 WatchSource:0}: Error finding container 440aa14cdaa59d43e8c78054395cda560f1246789b89b8a96af68a3cc0485776: Status 404 returned error can't find the container with id 440aa14cdaa59d43e8c78054395cda560f1246789b89b8a96af68a3cc0485776
Sep 17 15:59:39.495795 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:39.495762    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55bb4bf8d9-sthj9" event={"ID":"e7a2da14-69c7-40fc-86ac-ce5b85ca836f","Type":"ContainerStarted","Data":"6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b"}
Sep 17 15:59:39.496151 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:39.495816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55bb4bf8d9-sthj9" event={"ID":"e7a2da14-69c7-40fc-86ac-ce5b85ca836f","Type":"ContainerStarted","Data":"440aa14cdaa59d43e8c78054395cda560f1246789b89b8a96af68a3cc0485776"}
Sep 17 15:59:39.515508 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:39.515461    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55bb4bf8d9-sthj9" podStartSLOduration=1.5154455279999999 podStartE2EDuration="1.515445528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:59:39.513666858 +0000 UTC m=+328.950343638" watchObservedRunningTime="2026-09-17 15:59:39.515445528 +0000 UTC m=+328.952122307"
Sep 17 15:59:48.745242 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:48.745204    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:48.745721 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:48.745264    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:48.749947 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:48.749924    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:49.547815 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:49.547785    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 15:59:49.599044 ip-10-0-131-144 kubenswrapper[2566]: I0917 15:59:49.599004    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7df77c8b88-j4wf7"]
Sep 17 16:00:14.618329 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.618239    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7df77c8b88-j4wf7" podUID="ab946318-0b50-45d2-b6cb-d6d340b73272" containerName="console" containerID="cri-o://e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2" gracePeriod=15
Sep 17 16:00:14.863461 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.863437    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 16:00:14.869377 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869327    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-oauth-serving-cert\") pod \"ab946318-0b50-45d2-b6cb-d6d340b73272\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") "
Sep 17 16:00:14.869538 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869374    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-trusted-ca-bundle\") pod \"ab946318-0b50-45d2-b6cb-d6d340b73272\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") "
Sep 17 16:00:14.869538 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869438    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-serving-cert\") pod \"ab946318-0b50-45d2-b6cb-d6d340b73272\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") "
Sep 17 16:00:14.869538 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869471    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-console-config\") pod \"ab946318-0b50-45d2-b6cb-d6d340b73272\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") "
Sep 17 16:00:14.869538 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869502    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-service-ca\") pod \"ab946318-0b50-45d2-b6cb-d6d340b73272\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") "
Sep 17 16:00:14.869732 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869542    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnhkf\" (UniqueName: \"kubernetes.io/projected/ab946318-0b50-45d2-b6cb-d6d340b73272-kube-api-access-xnhkf\") pod \"ab946318-0b50-45d2-b6cb-d6d340b73272\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") "
Sep 17 16:00:14.869732 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869565    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-oauth-config\") pod \"ab946318-0b50-45d2-b6cb-d6d340b73272\" (UID: \"ab946318-0b50-45d2-b6cb-d6d340b73272\") "
Sep 17 16:00:14.869826 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869767    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ab946318-0b50-45d2-b6cb-d6d340b73272" (UID: "ab946318-0b50-45d2-b6cb-d6d340b73272"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:00:14.869876 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869821    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ab946318-0b50-45d2-b6cb-d6d340b73272" (UID: "ab946318-0b50-45d2-b6cb-d6d340b73272"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:00:14.869876 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869829    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-service-ca" (OuterVolumeSpecName: "service-ca") pod "ab946318-0b50-45d2-b6cb-d6d340b73272" (UID: "ab946318-0b50-45d2-b6cb-d6d340b73272"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:00:14.869876 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869838    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-console-config" (OuterVolumeSpecName: "console-config") pod "ab946318-0b50-45d2-b6cb-d6d340b73272" (UID: "ab946318-0b50-45d2-b6cb-d6d340b73272"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:00:14.869876 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.869868    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-oauth-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:00:14.871542 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.871518    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ab946318-0b50-45d2-b6cb-d6d340b73272" (UID: "ab946318-0b50-45d2-b6cb-d6d340b73272"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:00:14.871652 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.871566    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ab946318-0b50-45d2-b6cb-d6d340b73272" (UID: "ab946318-0b50-45d2-b6cb-d6d340b73272"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:00:14.871652 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.871595    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab946318-0b50-45d2-b6cb-d6d340b73272-kube-api-access-xnhkf" (OuterVolumeSpecName: "kube-api-access-xnhkf") pod "ab946318-0b50-45d2-b6cb-d6d340b73272" (UID: "ab946318-0b50-45d2-b6cb-d6d340b73272"). InnerVolumeSpecName "kube-api-access-xnhkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:00:14.971265 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.971235    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-trusted-ca-bundle\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:00:14.971265 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.971260    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:00:14.971265 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.971270    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-console-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:00:14.971463 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.971278    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab946318-0b50-45d2-b6cb-d6d340b73272-service-ca\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:00:14.971463 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.971286    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnhkf\" (UniqueName: \"kubernetes.io/projected/ab946318-0b50-45d2-b6cb-d6d340b73272-kube-api-access-xnhkf\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:00:14.971463 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:14.971294    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab946318-0b50-45d2-b6cb-d6d340b73272-console-oauth-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:00:15.645300 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.645263    2566 generic.go:356] "Generic (PLEG): container finished" podID="ab946318-0b50-45d2-b6cb-d6d340b73272" containerID="e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2" exitCode=0
Sep 17 16:00:15.645300 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.645302    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df77c8b88-j4wf7"
Sep 17 16:00:15.645766 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.645345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df77c8b88-j4wf7" event={"ID":"ab946318-0b50-45d2-b6cb-d6d340b73272","Type":"ContainerDied","Data":"e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2"}
Sep 17 16:00:15.645766 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.645395    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df77c8b88-j4wf7" event={"ID":"ab946318-0b50-45d2-b6cb-d6d340b73272","Type":"ContainerDied","Data":"7da1abcb57231691b8c4636000f22d9438d93d160f8893e0b5d96d70ccbe9983"}
Sep 17 16:00:15.645766 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.645415    2566 scope.go:117] "RemoveContainer" containerID="e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2"
Sep 17 16:00:15.653971 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.653954    2566 scope.go:117] "RemoveContainer" containerID="e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2"
Sep 17 16:00:15.654269 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:00:15.654250    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2\": container with ID starting with e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2 not found: ID does not exist" containerID="e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2"
Sep 17 16:00:15.654338 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.654278    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2"} err="failed to get container status \"e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2\": rpc error: code = NotFound desc = could not find container \"e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2\": container with ID starting with e1a3d2cb545dc58a62ce459b1534766f8222665c4a6b6d6252314677e6bae0c2 not found: ID does not exist"
Sep 17 16:00:15.660720 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.660697    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7df77c8b88-j4wf7"]
Sep 17 16:00:15.664605 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:15.664587    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7df77c8b88-j4wf7"]
Sep 17 16:00:17.047365 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:00:17.047330    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab946318-0b50-45d2-b6cb-d6d340b73272" path="/var/lib/kubelet/pods/ab946318-0b50-45d2-b6cb-d6d340b73272/volumes"
Sep 17 16:01:45.752051 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.751961    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"]
Sep 17 16:01:45.752638 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.752479    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab946318-0b50-45d2-b6cb-d6d340b73272" containerName="console"
Sep 17 16:01:45.752638 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.752499    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab946318-0b50-45d2-b6cb-d6d340b73272" containerName="console"
Sep 17 16:01:45.752638 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.752587    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab946318-0b50-45d2-b6cb-d6d340b73272" containerName="console"
Sep 17 16:01:45.755873 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.755851    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.758539 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.758519    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-57zjb\""
Sep 17 16:01:45.758639 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.758525    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 16:01:45.759542 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.759528    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 16:01:45.779659 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.779638    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"]
Sep 17 16:01:45.851207 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.851182    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.851322 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.851232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25pcf\" (UniqueName: \"kubernetes.io/projected/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-kube-api-access-25pcf\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.851322 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.851267    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.951754 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.951728    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25pcf\" (UniqueName: \"kubernetes.io/projected/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-kube-api-access-25pcf\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.951846 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.951767    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.951846 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.951832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.952277 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.952256    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.952277 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.952270    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:45.960749 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:45.960723    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25pcf\" (UniqueName: \"kubernetes.io/projected/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-kube-api-access-25pcf\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:46.064611 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:46.064556    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:01:46.335355 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:46.335329    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"]
Sep 17 16:01:46.337708 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:01:46.337669    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338423cb_4a13_4a9d_9c16_c3ce1c2e6cf5.slice/crio-91dc0033335974ad56f5504ee1581b6beca4567ecc58c701927867853d6be4ec WatchSource:0}: Error finding container 91dc0033335974ad56f5504ee1581b6beca4567ecc58c701927867853d6be4ec: Status 404 returned error can't find the container with id 91dc0033335974ad56f5504ee1581b6beca4567ecc58c701927867853d6be4ec
Sep 17 16:01:46.339581 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:46.339559    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 16:01:47.017355 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:47.017319    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j" event={"ID":"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5","Type":"ContainerStarted","Data":"91dc0033335974ad56f5504ee1581b6beca4567ecc58c701927867853d6be4ec"}
Sep 17 16:01:51.425269 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.425229    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bbff7bb64-nzknl"]
Sep 17 16:01:51.428749 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.428723    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.442583 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.442558    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbff7bb64-nzknl"]
Sep 17 16:01:51.502182 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.502130    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-config\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.502312 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.502203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-trusted-ca-bundle\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.502312 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.502244    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-oauth-config\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.502422 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.502323    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-serving-cert\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.502422 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.502360    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs84x\" (UniqueName: \"kubernetes.io/projected/2ea95922-956d-44ae-b4c4-7c0676eba8a9-kube-api-access-hs84x\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.502502 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.502423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-service-ca\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.502502 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.502460    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-oauth-serving-cert\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.603282 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.603249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-trusted-ca-bundle\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.603427 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.603289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-oauth-config\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.603427 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.603313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-serving-cert\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.603427 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.603331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs84x\" (UniqueName: \"kubernetes.io/projected/2ea95922-956d-44ae-b4c4-7c0676eba8a9-kube-api-access-hs84x\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.603591 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.603460    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-service-ca\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.603591 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.603516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-oauth-serving-cert\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.603591 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.603552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-config\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.604219 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.604196    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-config\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.604321 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.604217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-trusted-ca-bundle\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.604321 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.604257    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-service-ca\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.604321 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.604292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ea95922-956d-44ae-b4c4-7c0676eba8a9-oauth-serving-cert\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.605809 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.605790    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-oauth-config\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.605982 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.605964    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ea95922-956d-44ae-b4c4-7c0676eba8a9-console-serving-cert\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.611763 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.611745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs84x\" (UniqueName: \"kubernetes.io/projected/2ea95922-956d-44ae-b4c4-7c0676eba8a9-kube-api-access-hs84x\") pod \"console-bbff7bb64-nzknl\" (UID: \"2ea95922-956d-44ae-b4c4-7c0676eba8a9\") " pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:51.738576 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:51.738546    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:01:52.016713 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:52.016684    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbff7bb64-nzknl"]
Sep 17 16:01:52.018576 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:01:52.018545    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea95922_956d_44ae_b4c4_7c0676eba8a9.slice/crio-02cd16288a132739d6003537b4445b25f54198b44cdd1063609ed7fc3557ffea WatchSource:0}: Error finding container 02cd16288a132739d6003537b4445b25f54198b44cdd1063609ed7fc3557ffea: Status 404 returned error can't find the container with id 02cd16288a132739d6003537b4445b25f54198b44cdd1063609ed7fc3557ffea
Sep 17 16:01:52.040801 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:52.040773    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbff7bb64-nzknl" event={"ID":"2ea95922-956d-44ae-b4c4-7c0676eba8a9","Type":"ContainerStarted","Data":"02cd16288a132739d6003537b4445b25f54198b44cdd1063609ed7fc3557ffea"}
Sep 17 16:01:53.048207 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:53.048175    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbff7bb64-nzknl" event={"ID":"2ea95922-956d-44ae-b4c4-7c0676eba8a9","Type":"ContainerStarted","Data":"5ed45a95524c9f7816544f272d0cbac28872f1dc0115a43523c3015c19119dde"}
Sep 17 16:01:53.070414 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:53.070357    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bbff7bb64-nzknl" podStartSLOduration=2.070342346 podStartE2EDuration="2.070342346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:01:53.068105979 +0000 UTC m=+462.504782757" watchObservedRunningTime="2026-09-17 16:01:53.070342346 +0000 UTC m=+462.507019124"
Sep 17 16:01:56.064227 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:56.064188    2566 generic.go:356] "Generic (PLEG): container finished" podID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerID="cae1b6b2d1e012ae5f10948098a86f376e30880848c1b80412eb021caf733559" exitCode=0
Sep 17 16:01:56.064676 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:56.064274    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j" event={"ID":"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5","Type":"ContainerDied","Data":"cae1b6b2d1e012ae5f10948098a86f376e30880848c1b80412eb021caf733559"}
Sep 17 16:01:58.075371 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:58.075339    2566 generic.go:356] "Generic (PLEG): container finished" podID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerID="37d75901150833cbf0ceff002b4716b9898376892bed58bb90108ba4df7f750a" exitCode=0
Sep 17 16:01:58.075695 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:01:58.075423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j" event={"ID":"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5","Type":"ContainerDied","Data":"37d75901150833cbf0ceff002b4716b9898376892bed58bb90108ba4df7f750a"}
Sep 17 16:02:01.739589 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:01.739553    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:02:01.740057 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:01.739619    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:02:01.745020 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:01.744996    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:02:02.096068 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:02.095991    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bbff7bb64-nzknl"
Sep 17 16:02:02.157602 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:02.157571    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55bb4bf8d9-sthj9"]
Sep 17 16:02:06.112409 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:06.112368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j" event={"ID":"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5","Type":"ContainerStarted","Data":"618f243da29e537c06cd34ca64287f1eff06e41728e8bc1aae48d3090f0f8bde"}
Sep 17 16:02:06.130960 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:06.130911    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j" podStartSLOduration=2.528142768 podStartE2EDuration="21.130892399s" totalImagesPullingTime="18.602749631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:01:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:02:06.129758587 +0000 UTC m=+475.566435369" watchObservedRunningTime="2026-09-17 16:02:06.130892399 +0000 UTC m=+475.567569178"
Sep 17 16:02:07.117821 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:07.117784    2566 generic.go:356] "Generic (PLEG): container finished" podID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerID="618f243da29e537c06cd34ca64287f1eff06e41728e8bc1aae48d3090f0f8bde" exitCode=0
Sep 17 16:02:07.118199 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:07.117858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j" event={"ID":"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5","Type":"ContainerDied","Data":"618f243da29e537c06cd34ca64287f1eff06e41728e8bc1aae48d3090f0f8bde"}
Sep 17 16:02:08.292216 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.292187    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:02:08.355992 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.355966    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-util\") pod \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") "
Sep 17 16:02:08.356130 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.356018    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25pcf\" (UniqueName: \"kubernetes.io/projected/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-kube-api-access-25pcf\") pod \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") "
Sep 17 16:02:08.356130 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.356065    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-bundle\") pod \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\" (UID: \"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5\") "
Sep 17 16:02:08.356709 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.356688    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-bundle" (OuterVolumeSpecName: "bundle") pod "338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" (UID: "338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:02:08.358144 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.358121    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-kube-api-access-25pcf" (OuterVolumeSpecName: "kube-api-access-25pcf") pod "338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" (UID: "338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5"). InnerVolumeSpecName "kube-api-access-25pcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:02:08.363225 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.363203    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-util" (OuterVolumeSpecName: "util") pod "338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" (UID: "338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:02:08.457070 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.457019    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-util\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:08.457070 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.457042    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25pcf\" (UniqueName: \"kubernetes.io/projected/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-kube-api-access-25pcf\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:08.457070 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:08.457051    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5-bundle\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:09.127322 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:09.127292    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j"
Sep 17 16:02:09.127451 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:09.127321    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1apgl7j" event={"ID":"338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5","Type":"ContainerDied","Data":"91dc0033335974ad56f5504ee1581b6beca4567ecc58c701927867853d6be4ec"}
Sep 17 16:02:09.127451 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:09.127366    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91dc0033335974ad56f5504ee1581b6beca4567ecc58c701927867853d6be4ec"
Sep 17 16:02:13.751806 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.751772    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"]
Sep 17 16:02:13.752303 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.752282    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerName="util"
Sep 17 16:02:13.752380 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.752306    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerName="util"
Sep 17 16:02:13.752380 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.752341    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerName="extract"
Sep 17 16:02:13.752380 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.752350    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerName="extract"
Sep 17 16:02:13.752380 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.752372    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerName="pull"
Sep 17 16:02:13.752568 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.752382    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerName="pull"
Sep 17 16:02:13.752568 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.752461    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="338423cb-4a13-4a9d-9c16-c3ce1c2e6cf5" containerName="extract"
Sep 17 16:02:13.754562 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.754540    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:13.757601 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.757578    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 16:02:13.757693 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.757608    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 17 16:02:13.757693 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.757586    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 16:02:13.757798 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.757707    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-jlh86\""
Sep 17 16:02:13.770037 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.770014    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"]
Sep 17 16:02:13.900255 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.900232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f25d7156-559b-4efc-a536-0afdaefa99bc-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-g8nbq\" (UID: \"f25d7156-559b-4efc-a536-0afdaefa99bc\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:13.900386 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:13.900265    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwt6f\" (UniqueName: \"kubernetes.io/projected/f25d7156-559b-4efc-a536-0afdaefa99bc-kube-api-access-gwt6f\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-g8nbq\" (UID: \"f25d7156-559b-4efc-a536-0afdaefa99bc\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:14.000943 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:14.000909    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f25d7156-559b-4efc-a536-0afdaefa99bc-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-g8nbq\" (UID: \"f25d7156-559b-4efc-a536-0afdaefa99bc\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:14.000943 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:14.000950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwt6f\" (UniqueName: \"kubernetes.io/projected/f25d7156-559b-4efc-a536-0afdaefa99bc-kube-api-access-gwt6f\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-g8nbq\" (UID: \"f25d7156-559b-4efc-a536-0afdaefa99bc\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:14.003228 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:14.003179    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f25d7156-559b-4efc-a536-0afdaefa99bc-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-g8nbq\" (UID: \"f25d7156-559b-4efc-a536-0afdaefa99bc\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:14.009839 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:14.009818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwt6f\" (UniqueName: \"kubernetes.io/projected/f25d7156-559b-4efc-a536-0afdaefa99bc-kube-api-access-gwt6f\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-g8nbq\" (UID: \"f25d7156-559b-4efc-a536-0afdaefa99bc\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:14.064801 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:14.064779    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:14.343405 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:14.340609    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"]
Sep 17 16:02:14.346347 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:02:14.346311    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf25d7156_559b_4efc_a536_0afdaefa99bc.slice/crio-ab7bda99e9f6f875b44c53f6cbc32d51c0d74329c964507012cf15aa0d32c8a9 WatchSource:0}: Error finding container ab7bda99e9f6f875b44c53f6cbc32d51c0d74329c964507012cf15aa0d32c8a9: Status 404 returned error can't find the container with id ab7bda99e9f6f875b44c53f6cbc32d51c0d74329c964507012cf15aa0d32c8a9
Sep 17 16:02:15.158070 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:15.158032    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq" event={"ID":"f25d7156-559b-4efc-a536-0afdaefa99bc","Type":"ContainerStarted","Data":"ab7bda99e9f6f875b44c53f6cbc32d51c0d74329c964507012cf15aa0d32c8a9"}
Sep 17 16:02:17.168041 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.168011    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq" event={"ID":"f25d7156-559b-4efc-a536-0afdaefa99bc","Type":"ContainerStarted","Data":"f8863fa9da2ad3ec8d8ccbdc2c229f6edd9cfc08c5075c5b05b30c61e2f8f786"}
Sep 17 16:02:17.168497 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.168101    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:17.231041 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.230993    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq" podStartSLOduration=1.5153362769999998 podStartE2EDuration="4.230977067s" totalImagesPullingTime="2.71564079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:02:17.227682418 +0000 UTC m=+486.664359196" watchObservedRunningTime="2026-09-17 16:02:17.230977067 +0000 UTC m=+486.667653845"
Sep 17 16:02:17.781408 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.781373    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-gg9gk"]
Sep 17 16:02:17.784788 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.784768    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:17.788769 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.788738    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 16:02:17.788769 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.788750    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 17 16:02:17.788985 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.788966    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-c6frc\""
Sep 17 16:02:17.797059 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.797040    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-gg9gk"]
Sep 17 16:02:17.934188 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.934147    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:17.934363 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.934210    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s2bt\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-kube-api-access-7s2bt\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:17.934363 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.934251    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/047bc53b-030d-4c6e-82b1-2189e190022e-cabundle0\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:17.977774 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.977748    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"]
Sep 17 16:02:17.981067 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.981052    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:17.983436 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.983418    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 17 16:02:17.989801 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:17.989772    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"]
Sep 17 16:02:18.035107 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.035055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:18.035107 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.035104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s2bt\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-kube-api-access-7s2bt\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:18.035294 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.035142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/047bc53b-030d-4c6e-82b1-2189e190022e-cabundle0\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:18.035294 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.035210    2566 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Sep 17 16:02:18.035294 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.035239    2566 secret.go:281] references non-existent secret key: ca.crt
Sep 17 16:02:18.035294 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.035247    2566 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 16:02:18.035294 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.035259    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-gg9gk: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 16:02:18.035574 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.035313    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates podName:047bc53b-030d-4c6e-82b1-2189e190022e nodeName:}" failed. No retries permitted until 2026-09-17 16:02:18.535298174 +0000 UTC m=+487.971974946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates") pod "keda-operator-579c8958f7-gg9gk" (UID: "047bc53b-030d-4c6e-82b1-2189e190022e") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 16:02:18.035666 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.035649    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/047bc53b-030d-4c6e-82b1-2189e190022e-cabundle0\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:18.045911 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.045890    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s2bt\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-kube-api-access-7s2bt\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:18.135471 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.135444    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwsjb\" (UniqueName: \"kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-kube-api-access-dwsjb\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.135591 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.135565    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.135675 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.135659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-certificates\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.236095 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.236064    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.236506 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.236118    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-certificates\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.236506 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.236182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwsjb\" (UniqueName: \"kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-kube-api-access-dwsjb\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.236506 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.236292    2566 secret.go:281] references non-existent secret key: tls.crt
Sep 17 16:02:18.236506 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.236314    2566 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 17 16:02:18.236506 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.236337    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r: references non-existent secret key: tls.crt
Sep 17 16:02:18.236506 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.236389    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-certificates podName:a10d9ac5-cbdb-43d7-8604-4ae249b75e31 nodeName:}" failed. No retries permitted until 2026-09-17 16:02:18.736372685 +0000 UTC m=+488.173049456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-certificates") pod "keda-metrics-apiserver-77857b7b98-8k58r" (UID: "a10d9ac5-cbdb-43d7-8604-4ae249b75e31") : references non-existent secret key: tls.crt
Sep 17 16:02:18.236716 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.236591    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.244413 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.244389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwsjb\" (UniqueName: \"kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-kube-api-access-dwsjb\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.502125 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.502091    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-xgbwz"]
Sep 17 16:02:18.505733 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.505708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.508164 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.508135    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 17 16:02:18.516051 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.516026    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-xgbwz"]
Sep 17 16:02:18.538650 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.538627    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:18.538785 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.538769    2566 secret.go:281] references non-existent secret key: ca.crt
Sep 17 16:02:18.538835 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.538790    2566 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 16:02:18.538835 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.538798    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-gg9gk: references non-existent secret key: ca.crt
Sep 17 16:02:18.538904 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:18.538843    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates podName:047bc53b-030d-4c6e-82b1-2189e190022e nodeName:}" failed. No retries permitted until 2026-09-17 16:02:19.538829472 +0000 UTC m=+488.975506229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates") pod "keda-operator-579c8958f7-gg9gk" (UID: "047bc53b-030d-4c6e-82b1-2189e190022e") : references non-existent secret key: ca.crt
Sep 17 16:02:18.640014 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.639985    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxkst\" (UniqueName: \"kubernetes.io/projected/742acde5-4500-40af-aa02-05eb1ac87696-kube-api-access-xxkst\") pod \"keda-admission-6c4d864b86-xgbwz\" (UID: \"742acde5-4500-40af-aa02-05eb1ac87696\") " pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.640195 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.640131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/742acde5-4500-40af-aa02-05eb1ac87696-certificates\") pod \"keda-admission-6c4d864b86-xgbwz\" (UID: \"742acde5-4500-40af-aa02-05eb1ac87696\") " pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.741417 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.741384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxkst\" (UniqueName: \"kubernetes.io/projected/742acde5-4500-40af-aa02-05eb1ac87696-kube-api-access-xxkst\") pod \"keda-admission-6c4d864b86-xgbwz\" (UID: \"742acde5-4500-40af-aa02-05eb1ac87696\") " pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.741566 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.741427    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-certificates\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.741566 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.741488    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/742acde5-4500-40af-aa02-05eb1ac87696-certificates\") pod \"keda-admission-6c4d864b86-xgbwz\" (UID: \"742acde5-4500-40af-aa02-05eb1ac87696\") " pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.743995 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.743964    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a10d9ac5-cbdb-43d7-8604-4ae249b75e31-certificates\") pod \"keda-metrics-apiserver-77857b7b98-8k58r\" (UID: \"a10d9ac5-cbdb-43d7-8604-4ae249b75e31\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:18.744103 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.744026    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/742acde5-4500-40af-aa02-05eb1ac87696-certificates\") pod \"keda-admission-6c4d864b86-xgbwz\" (UID: \"742acde5-4500-40af-aa02-05eb1ac87696\") " pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.749751 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.749727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxkst\" (UniqueName: \"kubernetes.io/projected/742acde5-4500-40af-aa02-05eb1ac87696-kube-api-access-xxkst\") pod \"keda-admission-6c4d864b86-xgbwz\" (UID: \"742acde5-4500-40af-aa02-05eb1ac87696\") " pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.817606 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.817551    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:18.892222 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:18.892193    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:19.089859 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.089835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-xgbwz"]
Sep 17 16:02:19.092022 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:02:19.091994    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742acde5_4500_40af_aa02_05eb1ac87696.slice/crio-9c1c2d9f4023a8c9d3dc0fba815ec3dc7c97807247b19dcdbfff3aad4067e9f9 WatchSource:0}: Error finding container 9c1c2d9f4023a8c9d3dc0fba815ec3dc7c97807247b19dcdbfff3aad4067e9f9: Status 404 returned error can't find the container with id 9c1c2d9f4023a8c9d3dc0fba815ec3dc7c97807247b19dcdbfff3aad4067e9f9
Sep 17 16:02:19.165451 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.165391    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"]
Sep 17 16:02:19.167604 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:02:19.167580    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10d9ac5_cbdb_43d7_8604_4ae249b75e31.slice/crio-692ec728eb6a39c44a69c8fcec689594d1a8e0d4a9825d01c7ad1d6fda57d69c WatchSource:0}: Error finding container 692ec728eb6a39c44a69c8fcec689594d1a8e0d4a9825d01c7ad1d6fda57d69c: Status 404 returned error can't find the container with id 692ec728eb6a39c44a69c8fcec689594d1a8e0d4a9825d01c7ad1d6fda57d69c
Sep 17 16:02:19.179202 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.179148    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r" event={"ID":"a10d9ac5-cbdb-43d7-8604-4ae249b75e31","Type":"ContainerStarted","Data":"692ec728eb6a39c44a69c8fcec689594d1a8e0d4a9825d01c7ad1d6fda57d69c"}
Sep 17 16:02:19.180283 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.180260    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-xgbwz" event={"ID":"742acde5-4500-40af-aa02-05eb1ac87696","Type":"ContainerStarted","Data":"9c1c2d9f4023a8c9d3dc0fba815ec3dc7c97807247b19dcdbfff3aad4067e9f9"}
Sep 17 16:02:19.549887 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.549854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:19.552649 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.552626    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/047bc53b-030d-4c6e-82b1-2189e190022e-certificates\") pod \"keda-operator-579c8958f7-gg9gk\" (UID: \"047bc53b-030d-4c6e-82b1-2189e190022e\") " pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:19.594522 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.594487    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:19.888040 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:19.888009    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-gg9gk"]
Sep 17 16:02:19.890326 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:02:19.890286    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047bc53b_030d_4c6e_82b1_2189e190022e.slice/crio-32ea2bb4f73cfe38cf94863b4218a8a5084f1d066b8d97e90f12e2dceaeb5d84 WatchSource:0}: Error finding container 32ea2bb4f73cfe38cf94863b4218a8a5084f1d066b8d97e90f12e2dceaeb5d84: Status 404 returned error can't find the container with id 32ea2bb4f73cfe38cf94863b4218a8a5084f1d066b8d97e90f12e2dceaeb5d84
Sep 17 16:02:20.196862 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:20.196766    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-gg9gk" event={"ID":"047bc53b-030d-4c6e-82b1-2189e190022e","Type":"ContainerStarted","Data":"32ea2bb4f73cfe38cf94863b4218a8a5084f1d066b8d97e90f12e2dceaeb5d84"}
Sep 17 16:02:21.204329 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:21.204286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-xgbwz" event={"ID":"742acde5-4500-40af-aa02-05eb1ac87696","Type":"ContainerStarted","Data":"673d55e667c0ba6b20f686bdf49ce8799fbc970257c595704db93c45108b0e87"}
Sep 17 16:02:21.204790 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:21.204382    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:21.223514 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:21.223360    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-xgbwz" podStartSLOduration=1.96127103 podStartE2EDuration="3.223340439s" totalImagesPullingTime="1.262069409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:02:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:02:21.222169733 +0000 UTC m=+490.658846507" watchObservedRunningTime="2026-09-17 16:02:21.223340439 +0000 UTC m=+490.660017219"
Sep 17 16:02:22.211241 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:22.211192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r" event={"ID":"a10d9ac5-cbdb-43d7-8604-4ae249b75e31","Type":"ContainerStarted","Data":"a76fac964d08b84e4a7d60e30bb27e54beefe3df63b566b4860fb78a780ac02c"}
Sep 17 16:02:22.211696 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:22.211602    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:22.230185 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:22.230125    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r" podStartSLOduration=2.676930524 podStartE2EDuration="5.230106724s" totalImagesPullingTime="2.5531762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:02:22.229569578 +0000 UTC m=+491.666246384" watchObservedRunningTime="2026-09-17 16:02:22.230106724 +0000 UTC m=+491.666783505"
Sep 17 16:02:24.223405 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:24.223359    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-gg9gk" event={"ID":"047bc53b-030d-4c6e-82b1-2189e190022e","Type":"ContainerStarted","Data":"c11a5fd233562821d561b6e705ffa18700ccde104aaea30c08a7806a79071aa4"}
Sep 17 16:02:24.223817 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:24.223521    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:02:24.246411 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:24.246365    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-gg9gk" podStartSLOduration=3.844115105 podStartE2EDuration="7.246352277s" totalImagesPullingTime="3.402237172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:02:24.243824887 +0000 UTC m=+493.680501668" watchObservedRunningTime="2026-09-17 16:02:24.246352277 +0000 UTC m=+493.683029056"
Sep 17 16:02:27.185526 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.185484    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55bb4bf8d9-sthj9" podUID="e7a2da14-69c7-40fc-86ac-ce5b85ca836f" containerName="console" containerID="cri-o://6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b" gracePeriod=15
Sep 17 16:02:27.432749 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.432726    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 16:02:27.525973 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.525946    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-oauth-serving-cert\") pod \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") "
Sep 17 16:02:27.525973 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.525982    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-config\") pod \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") "
Sep 17 16:02:27.526194 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526035    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-serving-cert\") pod \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") "
Sep 17 16:02:27.526194 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526089    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-oauth-config\") pod \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") "
Sep 17 16:02:27.526194 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526107    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-trusted-ca-bundle\") pod \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") "
Sep 17 16:02:27.526194 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526132    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-service-ca\") pod \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") "
Sep 17 16:02:27.526194 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526181    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv8d5\" (UniqueName: \"kubernetes.io/projected/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-kube-api-access-nv8d5\") pod \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\" (UID: \"e7a2da14-69c7-40fc-86ac-ce5b85ca836f\") "
Sep 17 16:02:27.526475 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526434    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e7a2da14-69c7-40fc-86ac-ce5b85ca836f" (UID: "e7a2da14-69c7-40fc-86ac-ce5b85ca836f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:02:27.526475 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526460    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-config" (OuterVolumeSpecName: "console-config") pod "e7a2da14-69c7-40fc-86ac-ce5b85ca836f" (UID: "e7a2da14-69c7-40fc-86ac-ce5b85ca836f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:02:27.526865 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526832    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-service-ca" (OuterVolumeSpecName: "service-ca") pod "e7a2da14-69c7-40fc-86ac-ce5b85ca836f" (UID: "e7a2da14-69c7-40fc-86ac-ce5b85ca836f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:02:27.527009 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.526869    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e7a2da14-69c7-40fc-86ac-ce5b85ca836f" (UID: "e7a2da14-69c7-40fc-86ac-ce5b85ca836f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:02:27.528588 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.528561    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e7a2da14-69c7-40fc-86ac-ce5b85ca836f" (UID: "e7a2da14-69c7-40fc-86ac-ce5b85ca836f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:02:27.528681 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.528647    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-kube-api-access-nv8d5" (OuterVolumeSpecName: "kube-api-access-nv8d5") pod "e7a2da14-69c7-40fc-86ac-ce5b85ca836f" (UID: "e7a2da14-69c7-40fc-86ac-ce5b85ca836f"). InnerVolumeSpecName "kube-api-access-nv8d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:02:27.528740 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.528688    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e7a2da14-69c7-40fc-86ac-ce5b85ca836f" (UID: "e7a2da14-69c7-40fc-86ac-ce5b85ca836f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:02:27.627364 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.627328    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-oauth-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:27.627364 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.627362    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-trusted-ca-bundle\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:27.627553 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.627376    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-service-ca\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:27.627553 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.627388    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv8d5\" (UniqueName: \"kubernetes.io/projected/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-kube-api-access-nv8d5\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:27.627553 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.627401    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-oauth-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:27.627553 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.627411    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:27.627553 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:27.627423    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a2da14-69c7-40fc-86ac-ce5b85ca836f-console-serving-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:02:28.247210 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.247179    2566 generic.go:356] "Generic (PLEG): container finished" podID="e7a2da14-69c7-40fc-86ac-ce5b85ca836f" containerID="6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b" exitCode=0
Sep 17 16:02:28.247604 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.247244    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55bb4bf8d9-sthj9"
Sep 17 16:02:28.247604 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.247257    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55bb4bf8d9-sthj9" event={"ID":"e7a2da14-69c7-40fc-86ac-ce5b85ca836f","Type":"ContainerDied","Data":"6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b"}
Sep 17 16:02:28.247604 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.247303    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55bb4bf8d9-sthj9" event={"ID":"e7a2da14-69c7-40fc-86ac-ce5b85ca836f","Type":"ContainerDied","Data":"440aa14cdaa59d43e8c78054395cda560f1246789b89b8a96af68a3cc0485776"}
Sep 17 16:02:28.247604 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.247334    2566 scope.go:117] "RemoveContainer" containerID="6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b"
Sep 17 16:02:28.257458 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.257441    2566 scope.go:117] "RemoveContainer" containerID="6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b"
Sep 17 16:02:28.257731 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:02:28.257710    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b\": container with ID starting with 6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b not found: ID does not exist" containerID="6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b"
Sep 17 16:02:28.257797 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.257740    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b"} err="failed to get container status \"6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b\": rpc error: code = NotFound desc = could not find container \"6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b\": container with ID starting with 6c10d091b4079155449cda3148a555112af5ebd859b95b7c4f12ce429085493b not found: ID does not exist"
Sep 17 16:02:28.272413 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.272392    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55bb4bf8d9-sthj9"]
Sep 17 16:02:28.278713 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:28.278689    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55bb4bf8d9-sthj9"]
Sep 17 16:02:29.046044 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:29.046012    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7a2da14-69c7-40fc-86ac-ce5b85ca836f" path="/var/lib/kubelet/pods/e7a2da14-69c7-40fc-86ac-ce5b85ca836f/volumes"
Sep 17 16:02:34.228170 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:34.228128    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-8k58r"
Sep 17 16:02:38.176275 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:38.176246    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-g8nbq"
Sep 17 16:02:42.214656 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:42.214617    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-xgbwz"
Sep 17 16:02:45.233724 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:02:45.233695    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-gg9gk"
Sep 17 16:03:38.544862 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.544824    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"]
Sep 17 16:03:38.545270 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.545179    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7a2da14-69c7-40fc-86ac-ce5b85ca836f" containerName="console"
Sep 17 16:03:38.545270 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.545192    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7a2da14-69c7-40fc-86ac-ce5b85ca836f" containerName="console"
Sep 17 16:03:38.545270 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.545263    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7a2da14-69c7-40fc-86ac-ce5b85ca836f" containerName="console"
Sep 17 16:03:38.548151 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.548132    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:38.550847 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.550825    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-nt5rd\""
Sep 17 16:03:38.550937 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.550856    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 16:03:38.551522 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.551504    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 17 16:03:38.551522 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.551518    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 16:03:38.561847 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.561826    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"]
Sep 17 16:03:38.573287 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.573266    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-2vnns"]
Sep 17 16:03:38.576703 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.576684    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:38.579125 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.579104    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 17 16:03:38.579273 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.579107    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-swth9\""
Sep 17 16:03:38.585295 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.585274    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-2vnns"]
Sep 17 16:03:38.611477 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.611450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kcm5\" (UniqueName: \"kubernetes.io/projected/bd3e5381-7108-406a-b447-be83ea6dc841-kube-api-access-2kcm5\") pod \"seaweedfs-64568bcd49-2vnns\" (UID: \"bd3e5381-7108-406a-b447-be83ea6dc841\") " pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:38.611577 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.611518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmp2z\" (UniqueName: \"kubernetes.io/projected/0c963ef4-c5c2-4761-8abf-db8b9f21a460-kube-api-access-vmp2z\") pod \"llmisvc-controller-manager-645f9c8fd6-qsblw\" (UID: \"0c963ef4-c5c2-4761-8abf-db8b9f21a460\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:38.611577 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.611552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c963ef4-c5c2-4761-8abf-db8b9f21a460-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-qsblw\" (UID: \"0c963ef4-c5c2-4761-8abf-db8b9f21a460\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:38.611650 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.611576    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bd3e5381-7108-406a-b447-be83ea6dc841-data\") pod \"seaweedfs-64568bcd49-2vnns\" (UID: \"bd3e5381-7108-406a-b447-be83ea6dc841\") " pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:38.712846 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.712820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c963ef4-c5c2-4761-8abf-db8b9f21a460-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-qsblw\" (UID: \"0c963ef4-c5c2-4761-8abf-db8b9f21a460\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:38.712980 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.712856    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bd3e5381-7108-406a-b447-be83ea6dc841-data\") pod \"seaweedfs-64568bcd49-2vnns\" (UID: \"bd3e5381-7108-406a-b447-be83ea6dc841\") " pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:38.712980 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.712888    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcm5\" (UniqueName: \"kubernetes.io/projected/bd3e5381-7108-406a-b447-be83ea6dc841-kube-api-access-2kcm5\") pod \"seaweedfs-64568bcd49-2vnns\" (UID: \"bd3e5381-7108-406a-b447-be83ea6dc841\") " pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:38.712980 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.712934    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmp2z\" (UniqueName: \"kubernetes.io/projected/0c963ef4-c5c2-4761-8abf-db8b9f21a460-kube-api-access-vmp2z\") pod \"llmisvc-controller-manager-645f9c8fd6-qsblw\" (UID: \"0c963ef4-c5c2-4761-8abf-db8b9f21a460\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:38.712980 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:03:38.712971    2566 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 17 16:03:38.713116 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:03:38.713048    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c963ef4-c5c2-4761-8abf-db8b9f21a460-cert podName:0c963ef4-c5c2-4761-8abf-db8b9f21a460 nodeName:}" failed. No retries permitted until 2026-09-17 16:03:39.213029632 +0000 UTC m=+568.649706406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c963ef4-c5c2-4761-8abf-db8b9f21a460-cert") pod "llmisvc-controller-manager-645f9c8fd6-qsblw" (UID: "0c963ef4-c5c2-4761-8abf-db8b9f21a460") : secret "llmisvc-webhook-server-cert" not found
Sep 17 16:03:38.713295 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.713274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bd3e5381-7108-406a-b447-be83ea6dc841-data\") pod \"seaweedfs-64568bcd49-2vnns\" (UID: \"bd3e5381-7108-406a-b447-be83ea6dc841\") " pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:38.721793 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.721768    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmp2z\" (UniqueName: \"kubernetes.io/projected/0c963ef4-c5c2-4761-8abf-db8b9f21a460-kube-api-access-vmp2z\") pod \"llmisvc-controller-manager-645f9c8fd6-qsblw\" (UID: \"0c963ef4-c5c2-4761-8abf-db8b9f21a460\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:38.721862 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.721831    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kcm5\" (UniqueName: \"kubernetes.io/projected/bd3e5381-7108-406a-b447-be83ea6dc841-kube-api-access-2kcm5\") pod \"seaweedfs-64568bcd49-2vnns\" (UID: \"bd3e5381-7108-406a-b447-be83ea6dc841\") " pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:38.886505 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:38.886439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:39.156213 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:39.156188    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-2vnns"]
Sep 17 16:03:39.216616 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:39.216588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c963ef4-c5c2-4761-8abf-db8b9f21a460-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-qsblw\" (UID: \"0c963ef4-c5c2-4761-8abf-db8b9f21a460\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:39.218827 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:39.218799    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c963ef4-c5c2-4761-8abf-db8b9f21a460-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-qsblw\" (UID: \"0c963ef4-c5c2-4761-8abf-db8b9f21a460\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:39.458349 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:39.458268    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:39.584549 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:39.584508    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-2vnns" event={"ID":"bd3e5381-7108-406a-b447-be83ea6dc841","Type":"ContainerStarted","Data":"79c49c6c3ce09a9465628bc8d4aa30a8d311e89cbbc438449e17a8f9cf27c133"}
Sep 17 16:03:39.730732 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:39.730705    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"]
Sep 17 16:03:39.777221 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:03:39.777182    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c963ef4_c5c2_4761_8abf_db8b9f21a460.slice/crio-1a07655a5dc91d52b736477c5082da380afb52868385425638fa034919d47674 WatchSource:0}: Error finding container 1a07655a5dc91d52b736477c5082da380afb52868385425638fa034919d47674: Status 404 returned error can't find the container with id 1a07655a5dc91d52b736477c5082da380afb52868385425638fa034919d47674
Sep 17 16:03:40.592400 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:40.592323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw" event={"ID":"0c963ef4-c5c2-4761-8abf-db8b9f21a460","Type":"ContainerStarted","Data":"1a07655a5dc91d52b736477c5082da380afb52868385425638fa034919d47674"}
Sep 17 16:03:43.610135 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:43.610091    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-2vnns" event={"ID":"bd3e5381-7108-406a-b447-be83ea6dc841","Type":"ContainerStarted","Data":"6c6e2488fb1aab12242f81bffdfddf9d9de149d5a30e2162ff5f55c5b472d363"}
Sep 17 16:03:43.610645 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:43.610177    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:03:43.611654 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:43.611632    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw" event={"ID":"0c963ef4-c5c2-4761-8abf-db8b9f21a460","Type":"ContainerStarted","Data":"b7e51d0a3d016370bf2865b9da81a11420bb757973e8bc7ec7fe874aeb5ce215"}
Sep 17 16:03:43.611772 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:43.611754    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:03:43.627107 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:43.627064    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-2vnns" podStartSLOduration=1.457271679 podStartE2EDuration="5.627050639s" totalImagesPullingTime="4.16977896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:03:43.626328118 +0000 UTC m=+573.063004897" watchObservedRunningTime="2026-09-17 16:03:43.627050639 +0000 UTC m=+573.063727418"
Sep 17 16:03:43.644140 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:43.644092    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw" podStartSLOduration=2.136875996 podStartE2EDuration="5.644079795s" totalImagesPullingTime="3.507203799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:03:43.642876704 +0000 UTC m=+573.079553483" watchObservedRunningTime="2026-09-17 16:03:43.644079795 +0000 UTC m=+573.080756573"
Sep 17 16:03:49.619354 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:03:49.619321    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-2vnns"
Sep 17 16:04:11.022425 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:04:11.022390    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:04:11.025938 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:04:11.025914    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:04:11.030136 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:04:11.030115    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:04:11.033193 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:04:11.033178    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:04:14.618921 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:04:14.618890    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-qsblw"
Sep 17 16:06:44.961358 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.961328    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"]
Sep 17 16:06:44.964932 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.964913    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:44.967601 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.967580    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:06:44.967841 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.967820    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-z8dwv\""
Sep 17 16:06:44.967944 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.967841    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:06:44.967944 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.967913    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:06:44.968190 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.968173    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:06:44.968190 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.968183    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-v2-predictor-serving-cert\""
Sep 17 16:06:44.971010 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.970991    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Sep 17 16:06:44.980360 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:44.980334    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"]
Sep 17 16:06:45.016224 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.016201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.016317 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.016241    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zspz\" (UniqueName: \"kubernetes.io/projected/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-api-access-8zspz\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.016317 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.016273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.016387 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.016322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3113155a-b561-47a2-8cbd-8ded5bd5151a-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.016421 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.016397    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3113155a-b561-47a2-8cbd-8ded5bd5151a-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.117236 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3113155a-b561-47a2-8cbd-8ded5bd5151a-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.117356 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117248    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.117356 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117276    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zspz\" (UniqueName: \"kubernetes.io/projected/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-api-access-8zspz\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.117356 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.117356 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117343    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3113155a-b561-47a2-8cbd-8ded5bd5151a-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.117784 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117761    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3113155a-b561-47a2-8cbd-8ded5bd5151a-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.117944 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117924    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.118006 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.117979    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.119653 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.119633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3113155a-b561-47a2-8cbd-8ded5bd5151a-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.124873 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.124849    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zspz\" (UniqueName: \"kubernetes.io/projected/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-api-access-8zspz\") pod \"isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.275990 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.275963    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:06:45.550963 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:45.550937    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"]
Sep 17 16:06:45.552972 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:06:45.552940    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3113155a_b561_47a2_8cbd_8ded5bd5151a.slice/crio-c0fdbfc40e57be438dd27331ef06992a28bdc8452aabcb5aae4a7bec1e39d7ed WatchSource:0}: Error finding container c0fdbfc40e57be438dd27331ef06992a28bdc8452aabcb5aae4a7bec1e39d7ed: Status 404 returned error can't find the container with id c0fdbfc40e57be438dd27331ef06992a28bdc8452aabcb5aae4a7bec1e39d7ed
Sep 17 16:06:46.460562 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:46.460521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerStarted","Data":"c0fdbfc40e57be438dd27331ef06992a28bdc8452aabcb5aae4a7bec1e39d7ed"}
Sep 17 16:06:49.477977 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:49.477941    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerStarted","Data":"30f352e7a5a81664fc63e6c6cdf6ee75ea1a0869e8081afdab366455684c883c"}
Sep 17 16:06:53.497691 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:53.497656    2566 generic.go:356] "Generic (PLEG): container finished" podID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerID="30f352e7a5a81664fc63e6c6cdf6ee75ea1a0869e8081afdab366455684c883c" exitCode=0
Sep 17 16:06:53.498275 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:53.497704    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerDied","Data":"30f352e7a5a81664fc63e6c6cdf6ee75ea1a0869e8081afdab366455684c883c"}
Sep 17 16:06:53.498844 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:06:53.498824    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 16:07:07.588560 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:07.588516    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerStarted","Data":"c07991a10438e45e7237840b909740743acba3f10d610f779a00484c57c4b268"}
Sep 17 16:07:09.601723 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:09.601685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerStarted","Data":"75d41229cc4db3f101bb37630ae996822e7bf30ed2692393925efbfe5d5aeac0"}
Sep 17 16:07:09.602111 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:09.601750    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:07:09.602111 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:09.601885    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:07:09.603084 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:09.603058    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:07:09.640392 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:09.640348    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podStartSLOduration=2.807288147 podStartE2EDuration="25.6403348s" totalImagesPullingTime="18.971388968s" totalInitContainerRuntime="3.861657685s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:06:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:07:09.635735271 +0000 UTC m=+779.072412049" watchObservedRunningTime="2026-09-17 16:07:09.6403348 +0000 UTC m=+779.077011579"
Sep 17 16:07:10.607052 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:10.607005    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:07:15.612379 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:15.612344    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:07:15.612849 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:15.612816    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:07:25.613143 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:25.613099    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:07:35.613083 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:35.612993    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:07:45.613380 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:45.613336    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:07:55.613822 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:07:55.613777    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:08:05.613612 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:05.613569    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:08:15.612985 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:15.612942    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:08:25.613251 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:25.613218    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:08:34.700482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:34.700447    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"]
Sep 17 16:08:34.700898 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:34.700673    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" containerID="cri-o://c07991a10438e45e7237840b909740743acba3f10d610f779a00484c57c4b268" gracePeriod=30
Sep 17 16:08:34.700898 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:34.700717    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kube-rbac-proxy" containerID="cri-o://75d41229cc4db3f101bb37630ae996822e7bf30ed2692393925efbfe5d5aeac0" gracePeriod=30
Sep 17 16:08:35.016313 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:35.016280    2566 generic.go:356] "Generic (PLEG): container finished" podID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerID="75d41229cc4db3f101bb37630ae996822e7bf30ed2692393925efbfe5d5aeac0" exitCode=2
Sep 17 16:08:35.016462 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:35.016354    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerDied","Data":"75d41229cc4db3f101bb37630ae996822e7bf30ed2692393925efbfe5d5aeac0"}
Sep 17 16:08:35.607896 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:35.607850    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Sep 17 16:08:35.613273 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:35.613249    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 16:08:39.037723 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.037691    2566 generic.go:356] "Generic (PLEG): container finished" podID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerID="c07991a10438e45e7237840b909740743acba3f10d610f779a00484c57c4b268" exitCode=0
Sep 17 16:08:39.038061 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.037732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerDied","Data":"c07991a10438e45e7237840b909740743acba3f10d610f779a00484c57c4b268"}
Sep 17 16:08:39.055397 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.055374    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:08:39.175084 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175012    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zspz\" (UniqueName: \"kubernetes.io/projected/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-api-access-8zspz\") pod \"3113155a-b561-47a2-8cbd-8ded5bd5151a\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") "
Sep 17 16:08:39.175084 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175043    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3113155a-b561-47a2-8cbd-8ded5bd5151a-proxy-tls\") pod \"3113155a-b561-47a2-8cbd-8ded5bd5151a\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") "
Sep 17 16:08:39.175084 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175070    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3113155a-b561-47a2-8cbd-8ded5bd5151a-kserve-provision-location\") pod \"3113155a-b561-47a2-8cbd-8ded5bd5151a\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") "
Sep 17 16:08:39.175345 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175149    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-cabundle-cert\") pod \"3113155a-b561-47a2-8cbd-8ded5bd5151a\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") "
Sep 17 16:08:39.175345 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175192    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-rbac-proxy-sar-config\") pod \"3113155a-b561-47a2-8cbd-8ded5bd5151a\" (UID: \"3113155a-b561-47a2-8cbd-8ded5bd5151a\") "
Sep 17 16:08:39.175449 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175405    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3113155a-b561-47a2-8cbd-8ded5bd5151a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3113155a-b561-47a2-8cbd-8ded5bd5151a" (UID: "3113155a-b561-47a2-8cbd-8ded5bd5151a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:08:39.175601 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175574    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3113155a-b561-47a2-8cbd-8ded5bd5151a" (UID: "3113155a-b561-47a2-8cbd-8ded5bd5151a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:08:39.175674 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.175633    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3113155a-b561-47a2-8cbd-8ded5bd5151a" (UID: "3113155a-b561-47a2-8cbd-8ded5bd5151a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:08:39.177879 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.177858    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-api-access-8zspz" (OuterVolumeSpecName: "kube-api-access-8zspz") pod "3113155a-b561-47a2-8cbd-8ded5bd5151a" (UID: "3113155a-b561-47a2-8cbd-8ded5bd5151a"). InnerVolumeSpecName "kube-api-access-8zspz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:08:39.177879 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.177866    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3113155a-b561-47a2-8cbd-8ded5bd5151a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3113155a-b561-47a2-8cbd-8ded5bd5151a" (UID: "3113155a-b561-47a2-8cbd-8ded5bd5151a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:08:39.275847 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.275825    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:08:39.275954 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.275851    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:08:39.275954 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.275865    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zspz\" (UniqueName: \"kubernetes.io/projected/3113155a-b561-47a2-8cbd-8ded5bd5151a-kube-api-access-8zspz\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:08:39.275954 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.275879    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3113155a-b561-47a2-8cbd-8ded5bd5151a-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:08:39.275954 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.275891    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3113155a-b561-47a2-8cbd-8ded5bd5151a-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:08:39.895034 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895000    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"]
Sep 17 16:08:39.895413 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895399    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container"
Sep 17 16:08:39.895462 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895416    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container"
Sep 17 16:08:39.895462 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895437    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kube-rbac-proxy"
Sep 17 16:08:39.895462 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895443    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kube-rbac-proxy"
Sep 17 16:08:39.895462 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895457    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="storage-initializer"
Sep 17 16:08:39.895582 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895464    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="storage-initializer"
Sep 17 16:08:39.895582 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895524    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kserve-container"
Sep 17 16:08:39.895582 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.895531    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" containerName="kube-rbac-proxy"
Sep 17 16:08:39.898965 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.898947    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:39.901459 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.901436    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-4eea9-predictor-serving-cert\""
Sep 17 16:08:39.901550 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.901485    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-4eea9-kube-rbac-proxy-sar-config\""
Sep 17 16:08:39.910546 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.910524    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"]
Sep 17 16:08:39.981910 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.981880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-cabundle-cert\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:39.982030 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.981943    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18c94338-a626-413e-9ffa-fd024ee83003-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:39.982030 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.982001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:39.982107 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.982038    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:39.982107 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:39.982087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2kjq\" (UniqueName: \"kubernetes.io/projected/18c94338-a626-413e-9ffa-fd024ee83003-kube-api-access-b2kjq\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.045558 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.045531    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"
Sep 17 16:08:40.045912 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.045528    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s" event={"ID":"3113155a-b561-47a2-8cbd-8ded5bd5151a","Type":"ContainerDied","Data":"c0fdbfc40e57be438dd27331ef06992a28bdc8452aabcb5aae4a7bec1e39d7ed"}
Sep 17 16:08:40.045912 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.045662    2566 scope.go:117] "RemoveContainer" containerID="75d41229cc4db3f101bb37630ae996822e7bf30ed2692393925efbfe5d5aeac0"
Sep 17 16:08:40.049684 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.049661    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"]
Sep 17 16:08:40.053476 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.053459    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.055524 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.055457    2566 scope.go:117] "RemoveContainer" containerID="c07991a10438e45e7237840b909740743acba3f10d610f779a00484c57c4b268"
Sep 17 16:08:40.055990 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.055974    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-4eea9-predictor-serving-cert\""
Sep 17 16:08:40.056063 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.056016    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-4eea9-kube-rbac-proxy-sar-config\""
Sep 17 16:08:40.064436 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.064413    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"]
Sep 17 16:08:40.065447 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.065430    2566 scope.go:117] "RemoveContainer" containerID="30f352e7a5a81664fc63e6c6cdf6ee75ea1a0869e8081afdab366455684c883c"
Sep 17 16:08:40.083084 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083064    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2kjq\" (UniqueName: \"kubernetes.io/projected/18c94338-a626-413e-9ffa-fd024ee83003-kube-api-access-b2kjq\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.083215 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083106    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-cabundle-cert\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.083215 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083144    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18c94338-a626-413e-9ffa-fd024ee83003-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.083332 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.083332 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.083447 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:08:40.083366    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-serving-cert: secret "isvc-sklearn-graph-raw-4eea9-predictor-serving-cert" not found
Sep 17 16:08:40.083447 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:08:40.083429    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls podName:18c94338-a626-413e-9ffa-fd024ee83003 nodeName:}" failed. No retries permitted until 2026-09-17 16:08:40.583410854 +0000 UTC m=+870.020087628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls") pod "isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" (UID: "18c94338-a626-413e-9ffa-fd024ee83003") : secret "isvc-sklearn-graph-raw-4eea9-predictor-serving-cert" not found
Sep 17 16:08:40.083590 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18c94338-a626-413e-9ffa-fd024ee83003-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.083816 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083799    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-cabundle-cert\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.083886 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.083871    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.087949 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.087930    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"]
Sep 17 16:08:40.095584 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.095563    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2kjq\" (UniqueName: \"kubernetes.io/projected/18c94338-a626-413e-9ffa-fd024ee83003-kube-api-access-b2kjq\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.096060 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.096042    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-84f8f8b45-ffg2s"]
Sep 17 16:08:40.183776 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.183717    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-proxy-tls\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.183882 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.183785    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.183882 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.183866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-cabundle-cert\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.183992 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.183902    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.183992 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.183939    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46lkl\" (UniqueName: \"kubernetes.io/projected/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-api-access-46lkl\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.284550 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.284522    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-cabundle-cert\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.284708 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.284566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.284708 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.284601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46lkl\" (UniqueName: \"kubernetes.io/projected/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-api-access-46lkl\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.284708 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.284638    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-proxy-tls\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.284879 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.284836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.285042 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.285022    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.285362 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.285338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-cabundle-cert\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.285476 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.285460    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.286982 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.286963    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-proxy-tls\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.292747 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.292727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46lkl\" (UniqueName: \"kubernetes.io/projected/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-api-access-46lkl\") pod \"isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.366414 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.366391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:08:40.587599 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.587554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.589924 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.589904    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls\") pod \"isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:40.643912 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.643748    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"]
Sep 17 16:08:40.646591 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:08:40.646566    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2a4920_0110_4dfb_b75e_c35b6a3d30b2.slice/crio-ec0d90054c3b4da1214b39a38444aaef375bd14474a2332baf77e9f2ff4a9b48 WatchSource:0}: Error finding container ec0d90054c3b4da1214b39a38444aaef375bd14474a2332baf77e9f2ff4a9b48: Status 404 returned error can't find the container with id ec0d90054c3b4da1214b39a38444aaef375bd14474a2332baf77e9f2ff4a9b48
Sep 17 16:08:40.809392 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:40.809356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:41.052684 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:41.052641    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3113155a-b561-47a2-8cbd-8ded5bd5151a" path="/var/lib/kubelet/pods/3113155a-b561-47a2-8cbd-8ded5bd5151a/volumes"
Sep 17 16:08:41.055300 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:41.055271    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerStarted","Data":"3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2"}
Sep 17 16:08:41.055415 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:41.055329    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerStarted","Data":"ec0d90054c3b4da1214b39a38444aaef375bd14474a2332baf77e9f2ff4a9b48"}
Sep 17 16:08:41.084268 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:41.084246    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"]
Sep 17 16:08:41.086643 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:08:41.086617    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c94338_a626_413e_9ffa_fd024ee83003.slice/crio-6f088e16d06eac0b59d3ce6dbcd35ebe6bfe43eb1a39f5dfe0fa6216fbc359c8 WatchSource:0}: Error finding container 6f088e16d06eac0b59d3ce6dbcd35ebe6bfe43eb1a39f5dfe0fa6216fbc359c8: Status 404 returned error can't find the container with id 6f088e16d06eac0b59d3ce6dbcd35ebe6bfe43eb1a39f5dfe0fa6216fbc359c8
Sep 17 16:08:42.062579 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:42.062543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerStarted","Data":"aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159"}
Sep 17 16:08:42.062973 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:42.062599    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerStarted","Data":"6f088e16d06eac0b59d3ce6dbcd35ebe6bfe43eb1a39f5dfe0fa6216fbc359c8"}
Sep 17 16:08:45.078651 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:45.078621    2566 generic.go:356] "Generic (PLEG): container finished" podID="18c94338-a626-413e-9ffa-fd024ee83003" containerID="aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159" exitCode=0
Sep 17 16:08:45.079095 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:45.078690    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerDied","Data":"aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159"}
Sep 17 16:08:45.080397 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:45.080377    2566 generic.go:356] "Generic (PLEG): container finished" podID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerID="3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2" exitCode=0
Sep 17 16:08:45.080490 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:45.080451    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerDied","Data":"3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2"}
Sep 17 16:08:46.090171 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:46.090116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerStarted","Data":"bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae"}
Sep 17 16:08:46.090630 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:46.090242    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerStarted","Data":"884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401"}
Sep 17 16:08:46.090700 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:46.090671    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:46.091014 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:46.090731    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:46.092607 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:46.092577    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:08:46.112436 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:46.112391    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podStartSLOduration=3.223608593 podStartE2EDuration="7.112377501s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.888768908s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:08:46.110312685 +0000 UTC m=+875.546989457" watchObservedRunningTime="2026-09-17 16:08:46.112377501 +0000 UTC m=+875.549054280"
Sep 17 16:08:47.097539 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:47.097486    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:08:52.102568 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:52.102537    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:08:52.103266 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:08:52.103237    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:09:02.103436 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:02.103389    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:09:05.194377 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:05.194344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerStarted","Data":"3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d"}
Sep 17 16:09:05.194736 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:05.194407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerStarted","Data":"eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3"}
Sep 17 16:09:05.194736 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:05.194680    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:09:05.194857 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:05.194837    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:09:05.196205 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:05.196182    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:09:05.222961 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:05.222908    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podStartSLOduration=2.02568713 podStartE2EDuration="25.222895957s" totalImagesPullingTime="19.309032446s" totalInitContainerRuntime="3.888176381s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:09:05.220491694 +0000 UTC m=+894.657168474" watchObservedRunningTime="2026-09-17 16:09:05.222895957 +0000 UTC m=+894.659572735"
Sep 17 16:09:06.200600 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:06.200553    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:09:11.059995 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:11.059961    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:09:11.060615 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:11.060594    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:09:11.067711 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:11.067685    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:09:11.068406 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:11.068383    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:09:11.205592 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:11.205567    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:09:11.206085 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:11.206057    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:09:12.103825 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:12.103787    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:09:21.206740 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:21.206698    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:09:22.103469 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:22.103424    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:09:31.206698 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:31.206651    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:09:32.103454 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:32.103415    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:09:41.206891 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:41.206849    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:09:42.103528 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:42.103490    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:09:51.206531 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:51.206490    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:09:52.104048 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:09:52.104010    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:10:01.206768 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:01.206725    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:10:02.104188 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:02.104142    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:10:11.207240 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:11.207214    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:10:30.175951 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.175913    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"]
Sep 17 16:10:30.176556 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.176216    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" containerID="cri-o://884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401" gracePeriod=30
Sep 17 16:10:30.176556 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.176257    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kube-rbac-proxy" containerID="cri-o://bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae" gracePeriod=30
Sep 17 16:10:30.220275 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.220248    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"]
Sep 17 16:10:30.220497 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.220472    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" containerID="cri-o://eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3" gracePeriod=30
Sep 17 16:10:30.220574 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.220504    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kube-rbac-proxy" containerID="cri-o://3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d" gracePeriod=30
Sep 17 16:10:30.619579 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.619547    2566 generic.go:356] "Generic (PLEG): container finished" podID="18c94338-a626-413e-9ffa-fd024ee83003" containerID="bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae" exitCode=2
Sep 17 16:10:30.619759 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.619626    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerDied","Data":"bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae"}
Sep 17 16:10:30.621515 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.621486    2566 generic.go:356] "Generic (PLEG): container finished" podID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerID="3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d" exitCode=2
Sep 17 16:10:30.621643 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:30.621562    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerDied","Data":"3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d"}
Sep 17 16:10:31.201680 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:31.201639    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.38:8643/healthz\": dial tcp 10.133.0.38:8643: connect: connection refused"
Sep 17 16:10:31.206341 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:31.206317    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 16:10:32.098179 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:32.098124    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.37:8643/healthz\": dial tcp 10.133.0.37:8643: connect: connection refused"
Sep 17 16:10:32.103447 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:32.103419    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 16:10:34.162915 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.162885    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:10:34.261051 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261026    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46lkl\" (UniqueName: \"kubernetes.io/projected/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-api-access-46lkl\") pod \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") "
Sep 17 16:10:34.261225 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261109    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-proxy-tls\") pod \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") "
Sep 17 16:10:34.261225 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261152    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kserve-provision-location\") pod \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") "
Sep 17 16:10:34.261225 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261216    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-rbac-proxy-sar-config\") pod \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") "
Sep 17 16:10:34.261386 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261291    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-cabundle-cert\") pod \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\" (UID: \"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2\") "
Sep 17 16:10:34.261550 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261493    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" (UID: "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:10:34.261678 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261619    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" (UID: "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:10:34.261678 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.261634    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" (UID: "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:10:34.263231 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.263211    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" (UID: "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:10:34.263287 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.263274    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-api-access-46lkl" (OuterVolumeSpecName: "kube-api-access-46lkl") pod "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" (UID: "8f2a4920-0110-4dfb-b75e-c35b6a3d30b2"). InnerVolumeSpecName "kube-api-access-46lkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:10:34.362436 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.362404    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.362436 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.362429    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46lkl\" (UniqueName: \"kubernetes.io/projected/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-api-access-46lkl\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.362436 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.362441    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.362644 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.362450    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.362644 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.362458    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.608133 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.608108    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:10:34.642318 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.642289    2566 generic.go:356] "Generic (PLEG): container finished" podID="18c94338-a626-413e-9ffa-fd024ee83003" containerID="884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401" exitCode=0
Sep 17 16:10:34.642464 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.642344    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"
Sep 17 16:10:34.642464 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.642384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerDied","Data":"884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401"}
Sep 17 16:10:34.642464 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.642443    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s" event={"ID":"18c94338-a626-413e-9ffa-fd024ee83003","Type":"ContainerDied","Data":"6f088e16d06eac0b59d3ce6dbcd35ebe6bfe43eb1a39f5dfe0fa6216fbc359c8"}
Sep 17 16:10:34.642620 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.642486    2566 scope.go:117] "RemoveContainer" containerID="bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae"
Sep 17 16:10:34.645016 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.644991    2566 generic.go:356] "Generic (PLEG): container finished" podID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerID="eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3" exitCode=0
Sep 17 16:10:34.645150 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.645041    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerDied","Data":"eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3"}
Sep 17 16:10:34.645150 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.645085    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb" event={"ID":"8f2a4920-0110-4dfb-b75e-c35b6a3d30b2","Type":"ContainerDied","Data":"ec0d90054c3b4da1214b39a38444aaef375bd14474a2332baf77e9f2ff4a9b48"}
Sep 17 16:10:34.645150 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.645043    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"
Sep 17 16:10:34.651851 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.651828    2566 scope.go:117] "RemoveContainer" containerID="884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401"
Sep 17 16:10:34.660784 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.660764    2566 scope.go:117] "RemoveContainer" containerID="aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159"
Sep 17 16:10:34.667842 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.667825    2566 scope.go:117] "RemoveContainer" containerID="bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae"
Sep 17 16:10:34.668122 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:10:34.668102    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae\": container with ID starting with bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae not found: ID does not exist" containerID="bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae"
Sep 17 16:10:34.668192 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.668131    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae"} err="failed to get container status \"bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae\": rpc error: code = NotFound desc = could not find container \"bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae\": container with ID starting with bda19eac464e10e0c1aa52049c0f235fccad1e6b9453023364dd56f89745a8ae not found: ID does not exist"
Sep 17 16:10:34.668192 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.668168    2566 scope.go:117] "RemoveContainer" containerID="884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401"
Sep 17 16:10:34.668413 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:10:34.668395    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401\": container with ID starting with 884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401 not found: ID does not exist" containerID="884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401"
Sep 17 16:10:34.668459 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.668419    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401"} err="failed to get container status \"884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401\": rpc error: code = NotFound desc = could not find container \"884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401\": container with ID starting with 884d50d77ef179d2c9826cb04f8b486d53bf400da68e6e39f2c91a30969dc401 not found: ID does not exist"
Sep 17 16:10:34.668459 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.668432    2566 scope.go:117] "RemoveContainer" containerID="aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159"
Sep 17 16:10:34.668638 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:10:34.668619    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159\": container with ID starting with aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159 not found: ID does not exist" containerID="aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159"
Sep 17 16:10:34.668674 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.668641    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159"} err="failed to get container status \"aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159\": rpc error: code = NotFound desc = could not find container \"aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159\": container with ID starting with aa52952f7f65dd30b3d04d7613db34d6b4ba8df92051901640f13dda45e16159 not found: ID does not exist"
Sep 17 16:10:34.668674 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.668653    2566 scope.go:117] "RemoveContainer" containerID="3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d"
Sep 17 16:10:34.675546 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.675529    2566 scope.go:117] "RemoveContainer" containerID="eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3"
Sep 17 16:10:34.682437 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.682422    2566 scope.go:117] "RemoveContainer" containerID="3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2"
Sep 17 16:10:34.689432 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.689413    2566 scope.go:117] "RemoveContainer" containerID="3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d"
Sep 17 16:10:34.689710 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:10:34.689691    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d\": container with ID starting with 3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d not found: ID does not exist" containerID="3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d"
Sep 17 16:10:34.689788 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.689720    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d"} err="failed to get container status \"3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d\": rpc error: code = NotFound desc = could not find container \"3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d\": container with ID starting with 3a724aea28cd1e46e7a98712807f0ede2798f895906e6d9e0c187b93e618df3d not found: ID does not exist"
Sep 17 16:10:34.689788 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.689742    2566 scope.go:117] "RemoveContainer" containerID="eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3"
Sep 17 16:10:34.690024 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:10:34.690004    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3\": container with ID starting with eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3 not found: ID does not exist" containerID="eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3"
Sep 17 16:10:34.690063 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.690033    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3"} err="failed to get container status \"eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3\": rpc error: code = NotFound desc = could not find container \"eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3\": container with ID starting with eb863bf9cb2fbbe0e98f3d6c5659365ffec799b66dfee8e0719c1c1ceeda76d3 not found: ID does not exist"
Sep 17 16:10:34.690063 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.690047    2566 scope.go:117] "RemoveContainer" containerID="3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2"
Sep 17 16:10:34.690328 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:10:34.690298    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2\": container with ID starting with 3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2 not found: ID does not exist" containerID="3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2"
Sep 17 16:10:34.690392 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.690336    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2"} err="failed to get container status \"3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2\": rpc error: code = NotFound desc = could not find container \"3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2\": container with ID starting with 3a2f20923daa7864a1069c581f762cacaada3603d766705794568266b413bbe2 not found: ID does not exist"
Sep 17 16:10:34.691398 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.691380    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"]
Sep 17 16:10:34.709527 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.709507    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-4eea9-predictor-778c487dbf-pz2qb"]
Sep 17 16:10:34.764422 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764401    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-cabundle-cert\") pod \"18c94338-a626-413e-9ffa-fd024ee83003\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") "
Sep 17 16:10:34.764521 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764470    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2kjq\" (UniqueName: \"kubernetes.io/projected/18c94338-a626-413e-9ffa-fd024ee83003-kube-api-access-b2kjq\") pod \"18c94338-a626-413e-9ffa-fd024ee83003\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") "
Sep 17 16:10:34.764521 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764498    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-kube-rbac-proxy-sar-config\") pod \"18c94338-a626-413e-9ffa-fd024ee83003\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") "
Sep 17 16:10:34.764605 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764530    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18c94338-a626-413e-9ffa-fd024ee83003-kserve-provision-location\") pod \"18c94338-a626-413e-9ffa-fd024ee83003\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") "
Sep 17 16:10:34.764605 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764569    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls\") pod \"18c94338-a626-413e-9ffa-fd024ee83003\" (UID: \"18c94338-a626-413e-9ffa-fd024ee83003\") "
Sep 17 16:10:34.764814 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764785    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "18c94338-a626-413e-9ffa-fd024ee83003" (UID: "18c94338-a626-413e-9ffa-fd024ee83003"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:10:34.764887 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764842    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18c94338-a626-413e-9ffa-fd024ee83003-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "18c94338-a626-413e-9ffa-fd024ee83003" (UID: "18c94338-a626-413e-9ffa-fd024ee83003"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:10:34.764887 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.764869    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "18c94338-a626-413e-9ffa-fd024ee83003" (UID: "18c94338-a626-413e-9ffa-fd024ee83003"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:10:34.766596 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.766578    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c94338-a626-413e-9ffa-fd024ee83003-kube-api-access-b2kjq" (OuterVolumeSpecName: "kube-api-access-b2kjq") pod "18c94338-a626-413e-9ffa-fd024ee83003" (UID: "18c94338-a626-413e-9ffa-fd024ee83003"). InnerVolumeSpecName "kube-api-access-b2kjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:10:34.766675 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.766625    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "18c94338-a626-413e-9ffa-fd024ee83003" (UID: "18c94338-a626-413e-9ffa-fd024ee83003"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:10:34.865482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.865401    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18c94338-a626-413e-9ffa-fd024ee83003-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.865482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.865429    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18c94338-a626-413e-9ffa-fd024ee83003-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.865482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.865444    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.865482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.865455    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2kjq\" (UniqueName: \"kubernetes.io/projected/18c94338-a626-413e-9ffa-fd024ee83003-kube-api-access-b2kjq\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.865482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.865467    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18c94338-a626-413e-9ffa-fd024ee83003-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:10:34.970151 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.970113    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"]
Sep 17 16:10:34.976934 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:34.976908    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4eea9-predictor-7ff9769777-gr75s"]
Sep 17 16:10:35.046349 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:35.046316    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18c94338-a626-413e-9ffa-fd024ee83003" path="/var/lib/kubelet/pods/18c94338-a626-413e-9ffa-fd024ee83003/volumes"
Sep 17 16:10:35.046873 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:10:35.046854    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" path="/var/lib/kubelet/pods/8f2a4920-0110-4dfb-b75e-c35b6a3d30b2/volumes"
Sep 17 16:11:50.318857 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.318820    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"]
Sep 17 16:11:50.319442 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319381    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="storage-initializer"
Sep 17 16:11:50.319442 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319403    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="storage-initializer"
Sep 17 16:11:50.319442 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319419    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kube-rbac-proxy"
Sep 17 16:11:50.319442 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319428    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kube-rbac-proxy"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319445    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319454    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319469    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319479    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319503    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="storage-initializer"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319512    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="storage-initializer"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319526    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kube-rbac-proxy"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319535    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kube-rbac-proxy"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319620    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kserve-container"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319635    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kube-rbac-proxy"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319647    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c94338-a626-413e-9ffa-fd024ee83003" containerName="kserve-container"
Sep 17 16:11:50.319672 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.319657    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f2a4920-0110-4dfb-b75e-c35b6a3d30b2" containerName="kube-rbac-proxy"
Sep 17 16:11:50.323184 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.323163    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.325720 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.325697    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-68dcdd-predictor-serving-cert\""
Sep 17 16:11:50.325848 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.325725    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:11:50.325978 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.325962    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:11:50.326022 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.325965    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-z8dwv\""
Sep 17 16:11:50.326260 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.326244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:11:50.326605 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.326589    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:11:50.326683 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.326628    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-68dcdd-kube-rbac-proxy-sar-config\""
Sep 17 16:11:50.333537 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.333514    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"]
Sep 17 16:11:50.463422 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.463393    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-rbac-proxy-sar-config\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.463581 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.463444    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnvms\" (UniqueName: \"kubernetes.io/projected/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-api-access-wnvms\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.463581 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.463469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-cabundle-cert\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.463581 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.463497    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/133ae598-eec9-43e6-9c59-d62d25a5c10d-proxy-tls\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.463581 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.463513    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/133ae598-eec9-43e6-9c59-d62d25a5c10d-kserve-provision-location\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.564864 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.564835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnvms\" (UniqueName: \"kubernetes.io/projected/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-api-access-wnvms\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.565005 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.564874    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-cabundle-cert\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.565005 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.564951    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/133ae598-eec9-43e6-9c59-d62d25a5c10d-proxy-tls\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.565005 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.564984    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/133ae598-eec9-43e6-9c59-d62d25a5c10d-kserve-provision-location\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.565119 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.565028    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-rbac-proxy-sar-config\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.565477 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.565452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/133ae598-eec9-43e6-9c59-d62d25a5c10d-kserve-provision-location\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.565685 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.565669    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-rbac-proxy-sar-config\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.565740 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.565679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-cabundle-cert\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.567494 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.567472    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/133ae598-eec9-43e6-9c59-d62d25a5c10d-proxy-tls\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.574281 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.574211    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnvms\" (UniqueName: \"kubernetes.io/projected/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-api-access-wnvms\") pod \"isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") " pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.635031 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.634999    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:50.907964 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:50.907940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"]
Sep 17 16:11:50.909741 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:11:50.909709    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133ae598_eec9_43e6_9c59_d62d25a5c10d.slice/crio-ef935b6aabcca45632b77fb65c16ca4adc9fb98c49bfdc323247210a6c89ae7d WatchSource:0}: Error finding container ef935b6aabcca45632b77fb65c16ca4adc9fb98c49bfdc323247210a6c89ae7d: Status 404 returned error can't find the container with id ef935b6aabcca45632b77fb65c16ca4adc9fb98c49bfdc323247210a6c89ae7d
Sep 17 16:11:51.003751 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:51.003718    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerStarted","Data":"daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3"}
Sep 17 16:11:51.003879 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:51.003791    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerStarted","Data":"ef935b6aabcca45632b77fb65c16ca4adc9fb98c49bfdc323247210a6c89ae7d"}
Sep 17 16:11:55.025984 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:55.025950    2566 generic.go:356] "Generic (PLEG): container finished" podID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerID="daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3" exitCode=0
Sep 17 16:11:55.026342 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:55.026021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerDied","Data":"daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3"}
Sep 17 16:11:56.033136 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:56.033096    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerStarted","Data":"d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952"}
Sep 17 16:11:56.033551 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:56.033177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerStarted","Data":"f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053"}
Sep 17 16:11:56.033551 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:56.033389    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:56.066279 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:56.066231    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podStartSLOduration=2.20444464 podStartE2EDuration="6.066217899s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.861773259s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:11:56.064560868 +0000 UTC m=+1065.501237650" watchObservedRunningTime="2026-09-17 16:11:56.066217899 +0000 UTC m=+1065.502894678"
Sep 17 16:11:57.037941 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:57.037909    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:11:57.039624 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:57.039593    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:11:58.042667 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:11:58.042619    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:12:03.047602 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:12:03.047571    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:12:03.048119 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:12:03.048086    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:12:13.048862 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:12:13.048816    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:12:23.048052 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:12:23.048015    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:12:33.048884 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:12:33.048842    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:12:43.048473 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:12:43.048423    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:12:53.048260 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:12:53.048209    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:13:03.048404 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:03.048366    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 16:13:13.049221 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:13.049191    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:13:53.620410 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:53.620325    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"]
Sep 17 16:13:53.621019 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:53.620631    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container" containerID="cri-o://f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053" gracePeriod=30
Sep 17 16:13:53.621019 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:53.620703    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kube-rbac-proxy" containerID="cri-o://d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952" gracePeriod=30
Sep 17 16:13:54.591332 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:54.591298    2566 generic.go:356] "Generic (PLEG): container finished" podID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerID="d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952" exitCode=2
Sep 17 16:13:54.591504 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:54.591363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerDied","Data":"d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952"}
Sep 17 16:13:57.855420 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:57.855396    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:13:58.021734 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.021707    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnvms\" (UniqueName: \"kubernetes.io/projected/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-api-access-wnvms\") pod \"133ae598-eec9-43e6-9c59-d62d25a5c10d\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") "
Sep 17 16:13:58.021938 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.021744    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/133ae598-eec9-43e6-9c59-d62d25a5c10d-proxy-tls\") pod \"133ae598-eec9-43e6-9c59-d62d25a5c10d\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") "
Sep 17 16:13:58.021938 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.021790    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-rbac-proxy-sar-config\") pod \"133ae598-eec9-43e6-9c59-d62d25a5c10d\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") "
Sep 17 16:13:58.021938 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.021831    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-cabundle-cert\") pod \"133ae598-eec9-43e6-9c59-d62d25a5c10d\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") "
Sep 17 16:13:58.021938 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.021848    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/133ae598-eec9-43e6-9c59-d62d25a5c10d-kserve-provision-location\") pod \"133ae598-eec9-43e6-9c59-d62d25a5c10d\" (UID: \"133ae598-eec9-43e6-9c59-d62d25a5c10d\") "
Sep 17 16:13:58.022226 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.022191    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/133ae598-eec9-43e6-9c59-d62d25a5c10d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "133ae598-eec9-43e6-9c59-d62d25a5c10d" (UID: "133ae598-eec9-43e6-9c59-d62d25a5c10d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:13:58.022342 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.022232    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "133ae598-eec9-43e6-9c59-d62d25a5c10d" (UID: "133ae598-eec9-43e6-9c59-d62d25a5c10d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:13:58.022342 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.022237    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "133ae598-eec9-43e6-9c59-d62d25a5c10d" (UID: "133ae598-eec9-43e6-9c59-d62d25a5c10d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:13:58.023819 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.023800    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133ae598-eec9-43e6-9c59-d62d25a5c10d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "133ae598-eec9-43e6-9c59-d62d25a5c10d" (UID: "133ae598-eec9-43e6-9c59-d62d25a5c10d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:13:58.023819 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.023802    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-api-access-wnvms" (OuterVolumeSpecName: "kube-api-access-wnvms") pod "133ae598-eec9-43e6-9c59-d62d25a5c10d" (UID: "133ae598-eec9-43e6-9c59-d62d25a5c10d"). InnerVolumeSpecName "kube-api-access-wnvms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:13:58.122742 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.122718    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:13:58.122742 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.122739    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/133ae598-eec9-43e6-9c59-d62d25a5c10d-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:13:58.122885 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.122749    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnvms\" (UniqueName: \"kubernetes.io/projected/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-api-access-wnvms\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:13:58.122885 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.122759    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/133ae598-eec9-43e6-9c59-d62d25a5c10d-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:13:58.122885 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.122767    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/133ae598-eec9-43e6-9c59-d62d25a5c10d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:13:58.611681 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.611644    2566 generic.go:356] "Generic (PLEG): container finished" podID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerID="f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053" exitCode=0
Sep 17 16:13:58.611856 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.611703    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"
Sep 17 16:13:58.611856 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.611727    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerDied","Data":"f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053"}
Sep 17 16:13:58.611856 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.611787    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc" event={"ID":"133ae598-eec9-43e6-9c59-d62d25a5c10d","Type":"ContainerDied","Data":"ef935b6aabcca45632b77fb65c16ca4adc9fb98c49bfdc323247210a6c89ae7d"}
Sep 17 16:13:58.611856 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.611815    2566 scope.go:117] "RemoveContainer" containerID="d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952"
Sep 17 16:13:58.621188 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.621144    2566 scope.go:117] "RemoveContainer" containerID="f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053"
Sep 17 16:13:58.628576 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.628560    2566 scope.go:117] "RemoveContainer" containerID="daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3"
Sep 17 16:13:58.635869 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.635851    2566 scope.go:117] "RemoveContainer" containerID="d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952"
Sep 17 16:13:58.636120 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:13:58.636097    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952\": container with ID starting with d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952 not found: ID does not exist" containerID="d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952"
Sep 17 16:13:58.636192 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.636132    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952"} err="failed to get container status \"d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952\": rpc error: code = NotFound desc = could not find container \"d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952\": container with ID starting with d610bf3877740dff455873134bf022ce81f90115229a0615f252ea99a058b952 not found: ID does not exist"
Sep 17 16:13:58.636192 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.636148    2566 scope.go:117] "RemoveContainer" containerID="f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053"
Sep 17 16:13:58.636429 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:13:58.636393    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053\": container with ID starting with f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053 not found: ID does not exist" containerID="f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053"
Sep 17 16:13:58.636482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.636435    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053"} err="failed to get container status \"f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053\": rpc error: code = NotFound desc = could not find container \"f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053\": container with ID starting with f9958f898f3665d59f7f1501b12c3f2c86e097f8f84d6b3f00f8127014bea053 not found: ID does not exist"
Sep 17 16:13:58.636482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.636456    2566 scope.go:117] "RemoveContainer" containerID="daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3"
Sep 17 16:13:58.636740 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:13:58.636720    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3\": container with ID starting with daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3 not found: ID does not exist" containerID="daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3"
Sep 17 16:13:58.636796 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.636748    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3"} err="failed to get container status \"daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3\": rpc error: code = NotFound desc = could not find container \"daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3\": container with ID starting with daaa866f2fb6011df89fce738c0008b24531e5d3f1f7c4c5f6917fe3d26eaac3 not found: ID does not exist"
Sep 17 16:13:58.639181 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.639138    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"]
Sep 17 16:13:58.643735 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.643716    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-68dcdd-predictor-6f6d66c44-k5rgc"]
Sep 17 16:13:58.798813 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.798776    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"]
Sep 17 16:13:58.799213 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799195    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="storage-initializer"
Sep 17 16:13:58.799309 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799215    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="storage-initializer"
Sep 17 16:13:58.799309 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799231    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container"
Sep 17 16:13:58.799309 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799239    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container"
Sep 17 16:13:58.799309 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799284    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kube-rbac-proxy"
Sep 17 16:13:58.799309 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799293    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kube-rbac-proxy"
Sep 17 16:13:58.799564 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799380    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kserve-container"
Sep 17 16:13:58.799564 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.799395    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" containerName="kube-rbac-proxy"
Sep 17 16:13:58.808347 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.808313    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:58.811263 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.811235    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:13:58.811263 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.811235    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-init-fail-30d9d9-kube-rbac-proxy-sar-config\""
Sep 17 16:13:58.811523 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.811495    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fail-s3-secret-30d9d9\""
Sep 17 16:13:58.811661 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.811642    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:13:58.811734 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.811686    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:13:58.811819 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.811726    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-init-fail-30d9d9-predictor-serving-cert\""
Sep 17 16:13:58.811898 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.811848    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fail-s3-sa-30d9d9-dockercfg-w6hmp\""
Sep 17 16:13:58.813433 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.813410    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"]
Sep 17 16:13:58.928870 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.928801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:58.928870 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.928863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:58.929300 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.928904    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm7q7\" (UniqueName: \"kubernetes.io/projected/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-api-access-bm7q7\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:58.929300 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.928975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/820f0f60-ff1a-4737-87a8-d4a229b34f36-kserve-provision-location\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:58.929300 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:58.929039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-cabundle-cert\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.030304 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.030261    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.030304 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.030306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm7q7\" (UniqueName: \"kubernetes.io/projected/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-api-access-bm7q7\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.030487 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.030325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/820f0f60-ff1a-4737-87a8-d4a229b34f36-kserve-provision-location\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.030487 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:13:59.030411    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-serving-cert: secret "isvc-init-fail-30d9d9-predictor-serving-cert" not found
Sep 17 16:13:59.030487 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:13:59.030475    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls podName:820f0f60-ff1a-4737-87a8-d4a229b34f36 nodeName:}" failed. No retries permitted until 2026-09-17 16:13:59.530455866 +0000 UTC m=+1188.967132623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls") pod "isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" (UID: "820f0f60-ff1a-4737-87a8-d4a229b34f36") : secret "isvc-init-fail-30d9d9-predictor-serving-cert" not found
Sep 17 16:13:59.030623 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.030491    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-cabundle-cert\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.030623 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.030582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.030623 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.030603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/820f0f60-ff1a-4737-87a8-d4a229b34f36-kserve-provision-location\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.031201 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.031181    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-cabundle-cert\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.031292 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.031190    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.039612 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.039586    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm7q7\" (UniqueName: \"kubernetes.io/projected/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-api-access-bm7q7\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.046401 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.046380    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="133ae598-eec9-43e6-9c59-d62d25a5c10d" path="/var/lib/kubelet/pods/133ae598-eec9-43e6-9c59-d62d25a5c10d/volumes"
Sep 17 16:13:59.535116 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.535074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.537386 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.537369    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls\") pod \"isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") " pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.720303 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.720268    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:13:59.995931 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:13:59.995902    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"]
Sep 17 16:13:59.999803 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:13:59.999775    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820f0f60_ff1a_4737_87a8_d4a229b34f36.slice/crio-5abcc01df08666393fb2ad5cc021dd939b9235dfb26e6e6f8939726830d57a98 WatchSource:0}: Error finding container 5abcc01df08666393fb2ad5cc021dd939b9235dfb26e6e6f8939726830d57a98: Status 404 returned error can't find the container with id 5abcc01df08666393fb2ad5cc021dd939b9235dfb26e6e6f8939726830d57a98
Sep 17 16:14:00.624784 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:00.624749    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" event={"ID":"820f0f60-ff1a-4737-87a8-d4a229b34f36","Type":"ContainerStarted","Data":"4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62"}
Sep 17 16:14:00.624956 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:00.624809    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" event={"ID":"820f0f60-ff1a-4737-87a8-d4a229b34f36","Type":"ContainerStarted","Data":"5abcc01df08666393fb2ad5cc021dd939b9235dfb26e6e6f8939726830d57a98"}
Sep 17 16:14:02.635950 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:02.635926    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh_820f0f60-ff1a-4737-87a8-d4a229b34f36/storage-initializer/0.log"
Sep 17 16:14:02.636321 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:02.635968    2566 generic.go:356] "Generic (PLEG): container finished" podID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerID="4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62" exitCode=1
Sep 17 16:14:02.636321 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:02.636026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" event={"ID":"820f0f60-ff1a-4737-87a8-d4a229b34f36","Type":"ContainerDied","Data":"4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62"}
Sep 17 16:14:03.642568 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:03.642537    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh_820f0f60-ff1a-4737-87a8-d4a229b34f36/storage-initializer/0.log"
Sep 17 16:14:03.642943 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:03.642615    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" event={"ID":"820f0f60-ff1a-4737-87a8-d4a229b34f36","Type":"ContainerStarted","Data":"44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4"}
Sep 17 16:14:03.782329 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:03.782296    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"]
Sep 17 16:14:04.647850 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:04.647797    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerName="storage-initializer" containerID="cri-o://44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4" gracePeriod=30
Sep 17 16:14:08.295725 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.295704    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh_820f0f60-ff1a-4737-87a8-d4a229b34f36/storage-initializer/1.log"
Sep 17 16:14:08.296120 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.296103    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh_820f0f60-ff1a-4737-87a8-d4a229b34f36/storage-initializer/0.log"
Sep 17 16:14:08.298481 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.298468    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:14:08.406963 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.406904    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-rbac-proxy-sar-config\") pod \"820f0f60-ff1a-4737-87a8-d4a229b34f36\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") "
Sep 17 16:14:08.406963 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.406947    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/820f0f60-ff1a-4737-87a8-d4a229b34f36-kserve-provision-location\") pod \"820f0f60-ff1a-4737-87a8-d4a229b34f36\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") "
Sep 17 16:14:08.407126 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.406974    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls\") pod \"820f0f60-ff1a-4737-87a8-d4a229b34f36\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") "
Sep 17 16:14:08.407126 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.407003    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-cabundle-cert\") pod \"820f0f60-ff1a-4737-87a8-d4a229b34f36\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") "
Sep 17 16:14:08.407126 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.407050    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bm7q7\" (UniqueName: \"kubernetes.io/projected/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-api-access-bm7q7\") pod \"820f0f60-ff1a-4737-87a8-d4a229b34f36\" (UID: \"820f0f60-ff1a-4737-87a8-d4a229b34f36\") "
Sep 17 16:14:08.407314 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.407204    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/820f0f60-ff1a-4737-87a8-d4a229b34f36-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "820f0f60-ff1a-4737-87a8-d4a229b34f36" (UID: "820f0f60-ff1a-4737-87a8-d4a229b34f36"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:14:08.407314 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.407256    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "820f0f60-ff1a-4737-87a8-d4a229b34f36" (UID: "820f0f60-ff1a-4737-87a8-d4a229b34f36"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:14:08.407458 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.407435    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "820f0f60-ff1a-4737-87a8-d4a229b34f36" (UID: "820f0f60-ff1a-4737-87a8-d4a229b34f36"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:14:08.409017 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.408997    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-api-access-bm7q7" (OuterVolumeSpecName: "kube-api-access-bm7q7") pod "820f0f60-ff1a-4737-87a8-d4a229b34f36" (UID: "820f0f60-ff1a-4737-87a8-d4a229b34f36"). InnerVolumeSpecName "kube-api-access-bm7q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:14:08.409115 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.409096    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "820f0f60-ff1a-4737-87a8-d4a229b34f36" (UID: "820f0f60-ff1a-4737-87a8-d4a229b34f36"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:14:08.507986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.507956    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/820f0f60-ff1a-4737-87a8-d4a229b34f36-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:14:08.507986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.507984    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/820f0f60-ff1a-4737-87a8-d4a229b34f36-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:14:08.508141 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.507999    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:14:08.508141 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.508011    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm7q7\" (UniqueName: \"kubernetes.io/projected/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-api-access-bm7q7\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:14:08.508141 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.508023    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/820f0f60-ff1a-4737-87a8-d4a229b34f36-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:14:08.668001 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.667922    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh_820f0f60-ff1a-4737-87a8-d4a229b34f36/storage-initializer/1.log"
Sep 17 16:14:08.668321 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.668304    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh_820f0f60-ff1a-4737-87a8-d4a229b34f36/storage-initializer/0.log"
Sep 17 16:14:08.668381 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.668343    2566 generic.go:356] "Generic (PLEG): container finished" podID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerID="44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4" exitCode=1
Sep 17 16:14:08.668422 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.668410    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"
Sep 17 16:14:08.668455 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.668430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" event={"ID":"820f0f60-ff1a-4737-87a8-d4a229b34f36","Type":"ContainerDied","Data":"44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4"}
Sep 17 16:14:08.668498 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.668492    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh" event={"ID":"820f0f60-ff1a-4737-87a8-d4a229b34f36","Type":"ContainerDied","Data":"5abcc01df08666393fb2ad5cc021dd939b9235dfb26e6e6f8939726830d57a98"}
Sep 17 16:14:08.668532 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.668519    2566 scope.go:117] "RemoveContainer" containerID="44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4"
Sep 17 16:14:08.677409 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.677391    2566 scope.go:117] "RemoveContainer" containerID="4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62"
Sep 17 16:14:08.684959 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.684941    2566 scope.go:117] "RemoveContainer" containerID="44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4"
Sep 17 16:14:08.685247 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:14:08.685229    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4\": container with ID starting with 44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4 not found: ID does not exist" containerID="44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4"
Sep 17 16:14:08.685292 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.685257    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4"} err="failed to get container status \"44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4\": rpc error: code = NotFound desc = could not find container \"44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4\": container with ID starting with 44cb08277b28260d4eb47710010851ebadf569f22dc160c5a119bcc8e08836c4 not found: ID does not exist"
Sep 17 16:14:08.685292 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.685276    2566 scope.go:117] "RemoveContainer" containerID="4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62"
Sep 17 16:14:08.685527 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:14:08.685508    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62\": container with ID starting with 4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62 not found: ID does not exist" containerID="4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62"
Sep 17 16:14:08.685566 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.685532    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62"} err="failed to get container status \"4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62\": rpc error: code = NotFound desc = could not find container \"4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62\": container with ID starting with 4f8577c2aa7aaf81aa36842a2fdf934eb3909ff4ce7bc39b04b525ab4fb28d62 not found: ID does not exist"
Sep 17 16:14:08.708365 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.708311    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"]
Sep 17 16:14:08.710496 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:08.710474    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-30d9d9-predictor-5fb699fc7f-kmnqh"]
Sep 17 16:14:09.046058 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046020    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" path="/var/lib/kubelet/pods/820f0f60-ff1a-4737-87a8-d4a229b34f36/volumes"
Sep 17 16:14:09.046328 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046312    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"]
Sep 17 16:14:09.046648 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046635    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerName="storage-initializer"
Sep 17 16:14:09.046688 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046652    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerName="storage-initializer"
Sep 17 16:14:09.046739 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046730    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerName="storage-initializer"
Sep 17 16:14:09.046824 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046814    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerName="storage-initializer"
Sep 17 16:14:09.046857 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046824    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerName="storage-initializer"
Sep 17 16:14:09.046888 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.046870    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="820f0f60-ff1a-4737-87a8-d4a229b34f36" containerName="storage-initializer"
Sep 17 16:14:09.051202 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.051182    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.053660 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.053632    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:14:09.055215 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.055188    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:14:09.055215 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.055207    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-15cc8-kube-rbac-proxy-sar-config\""
Sep 17 16:14:09.055400 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.055195    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:14:09.055400 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.055302    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:14:09.055400 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.055305    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-15cc8-predictor-serving-cert\""
Sep 17 16:14:09.055535 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.055450    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-z8dwv\""
Sep 17 16:14:09.059023 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.059003    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"]
Sep 17 16:14:09.112971 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.112947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e58fe0ef-0700-46c8-aa29-fc369877c185-proxy-tls\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.113097 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.113001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e58fe0ef-0700-46c8-aa29-fc369877c185-kserve-provision-location\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.113097 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.113077    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-cabundle-cert\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.113260 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.113112    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jm25\" (UniqueName: \"kubernetes.io/projected/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-api-access-7jm25\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.113260 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.113201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.213783 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.213753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-cabundle-cert\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.213905 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.213785    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jm25\" (UniqueName: \"kubernetes.io/projected/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-api-access-7jm25\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.213905 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.213810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.213905 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.213836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e58fe0ef-0700-46c8-aa29-fc369877c185-proxy-tls\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.213905 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.213901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e58fe0ef-0700-46c8-aa29-fc369877c185-kserve-provision-location\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.214286 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.214263    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e58fe0ef-0700-46c8-aa29-fc369877c185-kserve-provision-location\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.214511 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.214490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-cabundle-cert\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.214576 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.214526    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.216327 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.216309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e58fe0ef-0700-46c8-aa29-fc369877c185-proxy-tls\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.222679 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.222654    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jm25\" (UniqueName: \"kubernetes.io/projected/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-api-access-7jm25\") pod \"raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") " pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.362364 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.362296    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:09.637299 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.637274    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"]
Sep 17 16:14:09.639800 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:14:09.639772    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58fe0ef_0700_46c8_aa29_fc369877c185.slice/crio-117be39267c6f2317981907521723ec5a11f58df97c522acc0aabc163de67152 WatchSource:0}: Error finding container 117be39267c6f2317981907521723ec5a11f58df97c522acc0aabc163de67152: Status 404 returned error can't find the container with id 117be39267c6f2317981907521723ec5a11f58df97c522acc0aabc163de67152
Sep 17 16:14:09.675483 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:09.675447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerStarted","Data":"117be39267c6f2317981907521723ec5a11f58df97c522acc0aabc163de67152"}
Sep 17 16:14:10.681435 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:10.681401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerStarted","Data":"f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62"}
Sep 17 16:14:11.093383 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:11.093352    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:14:11.093826 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:11.093805    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:14:11.100890 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:11.100865    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:14:11.101614 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:11.101598    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:14:14.701487 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:14.701455    2566 generic.go:356] "Generic (PLEG): container finished" podID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerID="f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62" exitCode=0
Sep 17 16:14:14.701857 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:14.701535    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerDied","Data":"f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62"}
Sep 17 16:14:15.707777 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:15.707743    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerStarted","Data":"0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54"}
Sep 17 16:14:15.708136 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:15.707808    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerStarted","Data":"1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4"}
Sep 17 16:14:15.708136 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:15.708014    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:15.733293 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:15.733249    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podStartSLOduration=2.596008952 podStartE2EDuration="6.733234966s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.137226014s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:14:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:14:15.732086812 +0000 UTC m=+1205.168763602" watchObservedRunningTime="2026-09-17 16:14:15.733234966 +0000 UTC m=+1205.169911746"
Sep 17 16:14:16.713139 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:16.713109    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:16.714656 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:16.714624    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:14:17.717956 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:17.717917    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:14:22.722645 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:22.722615    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:14:22.723191 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:22.723133    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:14:25.934507 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:25.934471    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"]
Sep 17 16:14:25.937887 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:25.937869    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:25.940428 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:25.940410    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-d5801-predictor-serving-cert\""
Sep 17 16:14:25.940535 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:25.940467    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-d5801-kube-rbac-proxy-sar-config\""
Sep 17 16:14:25.950901 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:25.950862    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"]
Sep 17 16:14:26.052388 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.052353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-cabundle-cert\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.052388 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.052388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78e999c8-d8e0-481a-8445-55b0ec8cd450-proxy-tls\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.052608 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.052412    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrj2q\" (UniqueName: \"kubernetes.io/projected/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-api-access-wrj2q\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.052608 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.052478    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.052608 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.052501    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78e999c8-d8e0-481a-8445-55b0ec8cd450-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.153661 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.153621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-cabundle-cert\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.153819 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.153675    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78e999c8-d8e0-481a-8445-55b0ec8cd450-proxy-tls\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.153819 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.153725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrj2q\" (UniqueName: \"kubernetes.io/projected/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-api-access-wrj2q\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.153819 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.153777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.153819 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.153802    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78e999c8-d8e0-481a-8445-55b0ec8cd450-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.154202 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.154176    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78e999c8-d8e0-481a-8445-55b0ec8cd450-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.154450 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.154424    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-cabundle-cert\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.154680 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.154661    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.156049 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.156022    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78e999c8-d8e0-481a-8445-55b0ec8cd450-proxy-tls\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.162516 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.162483    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrj2q\" (UniqueName: \"kubernetes.io/projected/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-api-access-wrj2q\") pod \"isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.248346 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.248318    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:26.525228 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.525203    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"]
Sep 17 16:14:26.527770 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:14:26.527738    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e999c8_d8e0_481a_8445_55b0ec8cd450.slice/crio-259a82caca41f5077e2e8bb94eb9b3c222b685bc5c5341e08c0b2f3bf78fcf7a WatchSource:0}: Error finding container 259a82caca41f5077e2e8bb94eb9b3c222b685bc5c5341e08c0b2f3bf78fcf7a: Status 404 returned error can't find the container with id 259a82caca41f5077e2e8bb94eb9b3c222b685bc5c5341e08c0b2f3bf78fcf7a
Sep 17 16:14:26.764692 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.764654    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerStarted","Data":"729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c"}
Sep 17 16:14:26.764877 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:26.764719    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerStarted","Data":"259a82caca41f5077e2e8bb94eb9b3c222b685bc5c5341e08c0b2f3bf78fcf7a"}
Sep 17 16:14:30.784980 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:30.784945    2566 generic.go:356] "Generic (PLEG): container finished" podID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerID="729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c" exitCode=0
Sep 17 16:14:30.785388 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:30.785024    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerDied","Data":"729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c"}
Sep 17 16:14:31.791779 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:31.791747    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerStarted","Data":"3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f"}
Sep 17 16:14:31.792176 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:31.791813    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerStarted","Data":"27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9"}
Sep 17 16:14:31.792176 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:31.792003    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:31.812905 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:31.812862    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podStartSLOduration=3.128167867 podStartE2EDuration="6.812849479s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.684681612s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:14:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:14:31.810710336 +0000 UTC m=+1221.247387114" watchObservedRunningTime="2026-09-17 16:14:31.812849479 +0000 UTC m=+1221.249526321"
Sep 17 16:14:32.723569 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:32.723531    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:14:32.796993 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:32.796964    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:32.798360 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:32.798331    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:14:33.802011 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:33.801969    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:14:38.806925 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:38.806893    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:14:38.807432 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:38.807401    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:14:42.723996 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:42.723950    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:14:48.808251 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:48.808208    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:14:52.723508 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:52.723470    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:14:58.807675 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:14:58.807629    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:15:02.723503 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:02.723458    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:15:08.808005 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:08.807962    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:15:12.723703 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:12.723662    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:15:18.807734 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:18.807695    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:15:22.724023 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:22.723981    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:15:28.807366 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:28.807326    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:15:32.724301 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:32.724268    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:15:38.808195 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:38.808131    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:15:39.212739 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:39.212657    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"]
Sep 17 16:15:39.212928 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:39.212883    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" containerID="cri-o://1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4" gracePeriod=30
Sep 17 16:15:39.213005 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:39.212916    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kube-rbac-proxy" containerID="cri-o://0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54" gracePeriod=30
Sep 17 16:15:40.139132 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:40.139098    2566 generic.go:356] "Generic (PLEG): container finished" podID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerID="0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54" exitCode=2
Sep 17 16:15:40.139497 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:40.139185    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerDied","Data":"0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54"}
Sep 17 16:15:42.718326 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:42.718281    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.41:8643/healthz\": dial tcp 10.133.0.41:8643: connect: connection refused"
Sep 17 16:15:42.723746 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:42.723715    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 16:15:43.662545 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.662520    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:15:43.823593 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.823566    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e58fe0ef-0700-46c8-aa29-fc369877c185-kserve-provision-location\") pod \"e58fe0ef-0700-46c8-aa29-fc369877c185\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") "
Sep 17 16:15:43.824028 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.823613    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-rbac-proxy-sar-config\") pod \"e58fe0ef-0700-46c8-aa29-fc369877c185\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") "
Sep 17 16:15:43.824028 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.823645    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e58fe0ef-0700-46c8-aa29-fc369877c185-proxy-tls\") pod \"e58fe0ef-0700-46c8-aa29-fc369877c185\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") "
Sep 17 16:15:43.824028 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.823684    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jm25\" (UniqueName: \"kubernetes.io/projected/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-api-access-7jm25\") pod \"e58fe0ef-0700-46c8-aa29-fc369877c185\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") "
Sep 17 16:15:43.824028 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.823705    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-cabundle-cert\") pod \"e58fe0ef-0700-46c8-aa29-fc369877c185\" (UID: \"e58fe0ef-0700-46c8-aa29-fc369877c185\") "
Sep 17 16:15:43.824028 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.823823    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fe0ef-0700-46c8-aa29-fc369877c185-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e58fe0ef-0700-46c8-aa29-fc369877c185" (UID: "e58fe0ef-0700-46c8-aa29-fc369877c185"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:15:43.824028 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.823902    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e58fe0ef-0700-46c8-aa29-fc369877c185-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:15:43.824288 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.824028    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e58fe0ef-0700-46c8-aa29-fc369877c185" (UID: "e58fe0ef-0700-46c8-aa29-fc369877c185"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:15:43.824288 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.824067    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e58fe0ef-0700-46c8-aa29-fc369877c185" (UID: "e58fe0ef-0700-46c8-aa29-fc369877c185"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:15:43.825721 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.825700    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-api-access-7jm25" (OuterVolumeSpecName: "kube-api-access-7jm25") pod "e58fe0ef-0700-46c8-aa29-fc369877c185" (UID: "e58fe0ef-0700-46c8-aa29-fc369877c185"). InnerVolumeSpecName "kube-api-access-7jm25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:15:43.825791 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.825719    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e58fe0ef-0700-46c8-aa29-fc369877c185-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e58fe0ef-0700-46c8-aa29-fc369877c185" (UID: "e58fe0ef-0700-46c8-aa29-fc369877c185"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:15:43.924455 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.924431    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jm25\" (UniqueName: \"kubernetes.io/projected/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-api-access-7jm25\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:15:43.924455 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.924456    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:15:43.924598 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.924466    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e58fe0ef-0700-46c8-aa29-fc369877c185-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:15:43.924598 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:43.924476    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e58fe0ef-0700-46c8-aa29-fc369877c185-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:15:44.160949 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.160865    2566 generic.go:356] "Generic (PLEG): container finished" podID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerID="1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4" exitCode=0
Sep 17 16:15:44.160949 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.160925    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"
Sep 17 16:15:44.161209 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.160947    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerDied","Data":"1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4"}
Sep 17 16:15:44.161209 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.161009    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6" event={"ID":"e58fe0ef-0700-46c8-aa29-fc369877c185","Type":"ContainerDied","Data":"117be39267c6f2317981907521723ec5a11f58df97c522acc0aabc163de67152"}
Sep 17 16:15:44.161209 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.161036    2566 scope.go:117] "RemoveContainer" containerID="0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54"
Sep 17 16:15:44.170246 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.170227    2566 scope.go:117] "RemoveContainer" containerID="1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4"
Sep 17 16:15:44.178256 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.178239    2566 scope.go:117] "RemoveContainer" containerID="f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62"
Sep 17 16:15:44.183968 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.183948    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"]
Sep 17 16:15:44.186331 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.186314    2566 scope.go:117] "RemoveContainer" containerID="0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54"
Sep 17 16:15:44.186611 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:15:44.186592    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54\": container with ID starting with 0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54 not found: ID does not exist" containerID="0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54"
Sep 17 16:15:44.186654 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.186620    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54"} err="failed to get container status \"0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54\": rpc error: code = NotFound desc = could not find container \"0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54\": container with ID starting with 0f0375450f5ee3642cc04b7fb6bffaf5d9cac56181607ed673b3beb73cfb2e54 not found: ID does not exist"
Sep 17 16:15:44.186654 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.186636    2566 scope.go:117] "RemoveContainer" containerID="1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4"
Sep 17 16:15:44.186863 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:15:44.186848    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4\": container with ID starting with 1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4 not found: ID does not exist" containerID="1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4"
Sep 17 16:15:44.186906 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.186867    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4"} err="failed to get container status \"1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4\": rpc error: code = NotFound desc = could not find container \"1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4\": container with ID starting with 1fdd2e5ca8fc7e771d17dd12605a23dd6044af292a64806905db2a185a7123e4 not found: ID does not exist"
Sep 17 16:15:44.186906 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.186880    2566 scope.go:117] "RemoveContainer" containerID="f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62"
Sep 17 16:15:44.187107 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:15:44.187090    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62\": container with ID starting with f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62 not found: ID does not exist" containerID="f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62"
Sep 17 16:15:44.187169 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.187113    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62"} err="failed to get container status \"f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62\": rpc error: code = NotFound desc = could not find container \"f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62\": container with ID starting with f7d5c647935fd9fbe9a5dbbe77dbc26d2d97d569752a88fc5d9f0559727f0d62 not found: ID does not exist"
Sep 17 16:15:44.187711 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:44.187693    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-15cc8-predictor-7bdc488bd5-h6db6"]
Sep 17 16:15:45.045089 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:45.045054    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" path="/var/lib/kubelet/pods/e58fe0ef-0700-46c8-aa29-fc369877c185/volumes"
Sep 17 16:15:48.807734 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:48.807695    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:15:58.807343 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:15:58.807297    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:16:02.039940 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:16:02.039899    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:16:12.040013 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:16:12.039966    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:16:22.039943 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:16:22.039891    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:16:32.040646 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:16:32.040597    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:16:42.040071 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:16:42.039979    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:16:52.039956 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:16:52.039906    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:17:02.041332 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:02.041304    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:17:06.047490 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:06.047455    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"]
Sep 17 16:17:06.047905 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:06.047721    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" containerID="cri-o://27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9" gracePeriod=30
Sep 17 16:17:06.047905 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:06.047776    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kube-rbac-proxy" containerID="cri-o://3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f" gracePeriod=30
Sep 17 16:17:06.566082 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:06.566048    2566 generic.go:356] "Generic (PLEG): container finished" podID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerID="3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f" exitCode=2
Sep 17 16:17:06.566266 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:06.566118    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerDied","Data":"3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f"}
Sep 17 16:17:08.803249 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:08.803210    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.42:8643/healthz\": dial tcp 10.133.0.42:8643: connect: connection refused"
Sep 17 16:17:12.040563 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:12.040523    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 16:17:13.802363 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:13.802326    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.42:8643/healthz\": dial tcp 10.133.0.42:8643: connect: connection refused"
Sep 17 16:17:15.303784 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.303758    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:17:15.371525 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371497    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-cabundle-cert\") pod \"78e999c8-d8e0-481a-8445-55b0ec8cd450\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") "
Sep 17 16:17:15.371693 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371532    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78e999c8-d8e0-481a-8445-55b0ec8cd450-proxy-tls\") pod \"78e999c8-d8e0-481a-8445-55b0ec8cd450\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") "
Sep 17 16:17:15.371693 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371556    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-rbac-proxy-sar-config\") pod \"78e999c8-d8e0-481a-8445-55b0ec8cd450\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") "
Sep 17 16:17:15.371693 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371575    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78e999c8-d8e0-481a-8445-55b0ec8cd450-kserve-provision-location\") pod \"78e999c8-d8e0-481a-8445-55b0ec8cd450\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") "
Sep 17 16:17:15.371693 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371605    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrj2q\" (UniqueName: \"kubernetes.io/projected/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-api-access-wrj2q\") pod \"78e999c8-d8e0-481a-8445-55b0ec8cd450\" (UID: \"78e999c8-d8e0-481a-8445-55b0ec8cd450\") "
Sep 17 16:17:15.371939 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371906    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e999c8-d8e0-481a-8445-55b0ec8cd450-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "78e999c8-d8e0-481a-8445-55b0ec8cd450" (UID: "78e999c8-d8e0-481a-8445-55b0ec8cd450"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:17:15.372002 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371939    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "78e999c8-d8e0-481a-8445-55b0ec8cd450" (UID: "78e999c8-d8e0-481a-8445-55b0ec8cd450"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:17:15.372002 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.371953    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "78e999c8-d8e0-481a-8445-55b0ec8cd450" (UID: "78e999c8-d8e0-481a-8445-55b0ec8cd450"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:17:15.373529 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.373507    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e999c8-d8e0-481a-8445-55b0ec8cd450-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "78e999c8-d8e0-481a-8445-55b0ec8cd450" (UID: "78e999c8-d8e0-481a-8445-55b0ec8cd450"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:17:15.373743 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.373725    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-api-access-wrj2q" (OuterVolumeSpecName: "kube-api-access-wrj2q") pod "78e999c8-d8e0-481a-8445-55b0ec8cd450" (UID: "78e999c8-d8e0-481a-8445-55b0ec8cd450"). InnerVolumeSpecName "kube-api-access-wrj2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:17:15.472657 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.472575    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:17:15.472657 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.472615    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78e999c8-d8e0-481a-8445-55b0ec8cd450-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:17:15.472657 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.472625    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:17:15.472657 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.472635    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78e999c8-d8e0-481a-8445-55b0ec8cd450-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:17:15.472657 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.472644    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrj2q\" (UniqueName: \"kubernetes.io/projected/78e999c8-d8e0-481a-8445-55b0ec8cd450-kube-api-access-wrj2q\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:17:15.607408 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.607377    2566 generic.go:356] "Generic (PLEG): container finished" podID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerID="27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9" exitCode=0
Sep 17 16:17:15.607546 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.607430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerDied","Data":"27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9"}
Sep 17 16:17:15.607546 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.607435    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"
Sep 17 16:17:15.607546 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.607477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4" event={"ID":"78e999c8-d8e0-481a-8445-55b0ec8cd450","Type":"ContainerDied","Data":"259a82caca41f5077e2e8bb94eb9b3c222b685bc5c5341e08c0b2f3bf78fcf7a"}
Sep 17 16:17:15.607546 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.607503    2566 scope.go:117] "RemoveContainer" containerID="3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f"
Sep 17 16:17:15.616437 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.616418    2566 scope.go:117] "RemoveContainer" containerID="27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9"
Sep 17 16:17:15.623784 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.623766    2566 scope.go:117] "RemoveContainer" containerID="729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c"
Sep 17 16:17:15.630056 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.630030    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"]
Sep 17 16:17:15.632291 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.632270    2566 scope.go:117] "RemoveContainer" containerID="3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f"
Sep 17 16:17:15.632576 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:17:15.632556    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f\": container with ID starting with 3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f not found: ID does not exist" containerID="3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f"
Sep 17 16:17:15.632667 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.632589    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f"} err="failed to get container status \"3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f\": rpc error: code = NotFound desc = could not find container \"3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f\": container with ID starting with 3c464168c829859964503df68057fa30103a454f5ffe2e0ba039ad50bfe90f7f not found: ID does not exist"
Sep 17 16:17:15.632667 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.632615    2566 scope.go:117] "RemoveContainer" containerID="27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9"
Sep 17 16:17:15.632863 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:17:15.632843    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9\": container with ID starting with 27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9 not found: ID does not exist" containerID="27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9"
Sep 17 16:17:15.632911 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.632868    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9"} err="failed to get container status \"27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9\": rpc error: code = NotFound desc = could not find container \"27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9\": container with ID starting with 27498f597809e06a109144de9c8e5b2256b5b0a8947b543453140af95b18d2b9 not found: ID does not exist"
Sep 17 16:17:15.632911 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.632889    2566 scope.go:117] "RemoveContainer" containerID="729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c"
Sep 17 16:17:15.633120 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:17:15.633102    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c\": container with ID starting with 729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c not found: ID does not exist" containerID="729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c"
Sep 17 16:17:15.633199 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.633127    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c"} err="failed to get container status \"729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c\": rpc error: code = NotFound desc = could not find container \"729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c\": container with ID starting with 729b41653bf5612494252279234ace83d6fa07a64c1d977cd2cba7c7d234a92c not found: ID does not exist"
Sep 17 16:17:15.633761 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:15.633742    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d5801-predictor-556d495c95-blmt4"]
Sep 17 16:17:17.045618 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:17:17.045584    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" path="/var/lib/kubelet/pods/78e999c8-d8e0-481a-8445-55b0ec8cd450/volumes"
Sep 17 16:18:46.552511 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552477    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"]
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552875    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="storage-initializer"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552890    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="storage-initializer"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552900    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="storage-initializer"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552906    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="storage-initializer"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552914    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552919    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552940    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552945    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552952    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kube-rbac-proxy"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552957    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kube-rbac-proxy"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552964    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kube-rbac-proxy"
Sep 17 16:18:46.552986 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.552969    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kube-rbac-proxy"
Sep 17 16:18:46.553425 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.553025    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kserve-container"
Sep 17 16:18:46.553425 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.553036    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kube-rbac-proxy"
Sep 17 16:18:46.553425 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.553042    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e999c8-d8e0-481a-8445-55b0ec8cd450" containerName="kserve-container"
Sep 17 16:18:46.553425 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.553048    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e58fe0ef-0700-46c8-aa29-fc369877c185" containerName="kube-rbac-proxy"
Sep 17 16:18:46.556346 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.556326    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.559076 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.559045    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:18:46.559218 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.559111    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:18:46.559218 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.559131    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-z8dwv\""
Sep 17 16:18:46.559218 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.559199    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:18:46.559719 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.559704    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:18:46.559789 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.559750    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Sep 17 16:18:46.559878 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.559861    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-predictor-serving-cert\""
Sep 17 16:18:46.567467 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.567446    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"]
Sep 17 16:18:46.585466 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.585439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46fbf3be-1291-41f0-86fb-a4636eb991f7-proxy-tls\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.585558 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.585500    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtz6q\" (UniqueName: \"kubernetes.io/projected/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-api-access-rtz6q\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.585598 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.585566    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-cabundle-cert\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.585634 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.585603    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.585675 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.585638    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46fbf3be-1291-41f0-86fb-a4636eb991f7-kserve-provision-location\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686193 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686308 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46fbf3be-1291-41f0-86fb-a4636eb991f7-kserve-provision-location\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686308 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46fbf3be-1291-41f0-86fb-a4636eb991f7-proxy-tls\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686410 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686336    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtz6q\" (UniqueName: \"kubernetes.io/projected/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-api-access-rtz6q\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686410 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-cabundle-cert\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686632 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686611    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46fbf3be-1291-41f0-86fb-a4636eb991f7-kserve-provision-location\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686865 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686844    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.686916 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.686868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-cabundle-cert\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.688722 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.688704    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46fbf3be-1291-41f0-86fb-a4636eb991f7-proxy-tls\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.694747 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.694727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtz6q\" (UniqueName: \"kubernetes.io/projected/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-api-access-rtz6q\") pod \"isvc-canary-promote-predictor-5fd5fb8776-8jhrd\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:46.866625 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:46.866561    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:47.151492 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:47.151434    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"]
Sep 17 16:18:47.153974 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:18:47.153952    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fbf3be_1291_41f0_86fb_a4636eb991f7.slice/crio-bf78ffde2d5f182b9ba1c29138126845d6d2de1aac8d2f0d4f2217747cc3c9f7 WatchSource:0}: Error finding container bf78ffde2d5f182b9ba1c29138126845d6d2de1aac8d2f0d4f2217747cc3c9f7: Status 404 returned error can't find the container with id bf78ffde2d5f182b9ba1c29138126845d6d2de1aac8d2f0d4f2217747cc3c9f7
Sep 17 16:18:48.035507 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:48.035464    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerStarted","Data":"48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7"}
Sep 17 16:18:48.035860 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:48.035525    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerStarted","Data":"bf78ffde2d5f182b9ba1c29138126845d6d2de1aac8d2f0d4f2217747cc3c9f7"}
Sep 17 16:18:52.057828 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:52.057792    2566 generic.go:356] "Generic (PLEG): container finished" podID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerID="48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7" exitCode=0
Sep 17 16:18:52.058255 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:52.057843    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerDied","Data":"48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7"}
Sep 17 16:18:53.065189 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:53.065142    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerStarted","Data":"0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0"}
Sep 17 16:18:53.065659 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:53.065227    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerStarted","Data":"a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e"}
Sep 17 16:18:53.065659 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:53.065444    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:53.086864 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:53.086822    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podStartSLOduration=3.113820932 podStartE2EDuration="7.086808915s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.972987983s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:18:53.085275732 +0000 UTC m=+1482.521952510" watchObservedRunningTime="2026-09-17 16:18:53.086808915 +0000 UTC m=+1482.523485695"
Sep 17 16:18:54.069598 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:54.069564    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:18:54.070849 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:54.070815    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:18:55.074352 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:18:55.074317    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:19:00.078858 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:00.078829    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:19:00.079395 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:00.079369    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:19:10.079710 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:10.079669    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:19:11.126776 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:11.126747    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:19:11.128743 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:11.128714    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:19:11.134049 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:11.134029    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:19:11.136227 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:11.136208    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:19:20.079513 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:20.079464    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:19:30.079527 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:30.079485    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:19:40.080207 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:40.080113    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:19:50.080210 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:19:50.080172    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:20:00.080306 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:00.080265    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 16:20:08.975308 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:08.975264    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"]
Sep 17 16:20:08.975760 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:08.975559    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container" containerID="cri-o://a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e" gracePeriod=30
Sep 17 16:20:08.975760 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:08.975616    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kube-rbac-proxy" containerID="cri-o://0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0" gracePeriod=30
Sep 17 16:20:08.976390 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:08.976337    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:20:09.426466 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:09.426435    2566 generic.go:356] "Generic (PLEG): container finished" podID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerID="0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0" exitCode=2
Sep 17 16:20:09.426625 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:09.426508    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerDied","Data":"0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0"}
Sep 17 16:20:10.075141 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:10.075098    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.43:8643/healthz\": dial tcp 10.133.0.43:8643: connect: connection refused"
Sep 17 16:20:13.227608 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.227583    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:20:13.317707 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.317626    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46fbf3be-1291-41f0-86fb-a4636eb991f7-proxy-tls\") pod \"46fbf3be-1291-41f0-86fb-a4636eb991f7\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") "
Sep 17 16:20:13.317707 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.317664    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-rbac-proxy-sar-config\") pod \"46fbf3be-1291-41f0-86fb-a4636eb991f7\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") "
Sep 17 16:20:13.317707 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.317703    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtz6q\" (UniqueName: \"kubernetes.io/projected/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-api-access-rtz6q\") pod \"46fbf3be-1291-41f0-86fb-a4636eb991f7\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") "
Sep 17 16:20:13.317967 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.317727    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46fbf3be-1291-41f0-86fb-a4636eb991f7-kserve-provision-location\") pod \"46fbf3be-1291-41f0-86fb-a4636eb991f7\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") "
Sep 17 16:20:13.317967 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.317779    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-cabundle-cert\") pod \"46fbf3be-1291-41f0-86fb-a4636eb991f7\" (UID: \"46fbf3be-1291-41f0-86fb-a4636eb991f7\") "
Sep 17 16:20:13.318134 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.318100    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46fbf3be-1291-41f0-86fb-a4636eb991f7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "46fbf3be-1291-41f0-86fb-a4636eb991f7" (UID: "46fbf3be-1291-41f0-86fb-a4636eb991f7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:20:13.318215 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.318127    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "46fbf3be-1291-41f0-86fb-a4636eb991f7" (UID: "46fbf3be-1291-41f0-86fb-a4636eb991f7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:20:13.318215 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.318189    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "46fbf3be-1291-41f0-86fb-a4636eb991f7" (UID: "46fbf3be-1291-41f0-86fb-a4636eb991f7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:20:13.319847 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.319824    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46fbf3be-1291-41f0-86fb-a4636eb991f7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "46fbf3be-1291-41f0-86fb-a4636eb991f7" (UID: "46fbf3be-1291-41f0-86fb-a4636eb991f7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:20:13.320077 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.320061    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-api-access-rtz6q" (OuterVolumeSpecName: "kube-api-access-rtz6q") pod "46fbf3be-1291-41f0-86fb-a4636eb991f7" (UID: "46fbf3be-1291-41f0-86fb-a4636eb991f7"). InnerVolumeSpecName "kube-api-access-rtz6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:20:13.418390 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.418363    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46fbf3be-1291-41f0-86fb-a4636eb991f7-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:20:13.418390 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.418389    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:20:13.418544 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.418403    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtz6q\" (UniqueName: \"kubernetes.io/projected/46fbf3be-1291-41f0-86fb-a4636eb991f7-kube-api-access-rtz6q\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:20:13.418544 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.418415    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46fbf3be-1291-41f0-86fb-a4636eb991f7-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:20:13.418544 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.418427    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46fbf3be-1291-41f0-86fb-a4636eb991f7-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:20:13.452767 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.452731    2566 generic.go:356] "Generic (PLEG): container finished" podID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerID="a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e" exitCode=0
Sep 17 16:20:13.452903 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.452784    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"
Sep 17 16:20:13.452903 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.452815    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerDied","Data":"a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e"}
Sep 17 16:20:13.452903 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.452877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd" event={"ID":"46fbf3be-1291-41f0-86fb-a4636eb991f7","Type":"ContainerDied","Data":"bf78ffde2d5f182b9ba1c29138126845d6d2de1aac8d2f0d4f2217747cc3c9f7"}
Sep 17 16:20:13.452903 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.452903    2566 scope.go:117] "RemoveContainer" containerID="0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0"
Sep 17 16:20:13.462278 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.462260    2566 scope.go:117] "RemoveContainer" containerID="a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e"
Sep 17 16:20:13.469652 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.469636    2566 scope.go:117] "RemoveContainer" containerID="48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7"
Sep 17 16:20:13.476454 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.476433    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"]
Sep 17 16:20:13.477050 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.477038    2566 scope.go:117] "RemoveContainer" containerID="0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0"
Sep 17 16:20:13.477329 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:20:13.477310    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0\": container with ID starting with 0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0 not found: ID does not exist" containerID="0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0"
Sep 17 16:20:13.477396 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.477338    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0"} err="failed to get container status \"0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0\": rpc error: code = NotFound desc = could not find container \"0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0\": container with ID starting with 0a73fe97f439819c2ab8ab53b4d3c556e9038ca7f5a4fe896c769963e19067f0 not found: ID does not exist"
Sep 17 16:20:13.477396 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.477354    2566 scope.go:117] "RemoveContainer" containerID="a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e"
Sep 17 16:20:13.477622 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:20:13.477605    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e\": container with ID starting with a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e not found: ID does not exist" containerID="a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e"
Sep 17 16:20:13.477662 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.477628    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e"} err="failed to get container status \"a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e\": rpc error: code = NotFound desc = could not find container \"a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e\": container with ID starting with a7b41a55ee9d24114bc34325ff7046522a410ee610cf956ef564c51ad51fd71e not found: ID does not exist"
Sep 17 16:20:13.477662 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.477645    2566 scope.go:117] "RemoveContainer" containerID="48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7"
Sep 17 16:20:13.477874 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:20:13.477859    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7\": container with ID starting with 48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7 not found: ID does not exist" containerID="48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7"
Sep 17 16:20:13.477918 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.477878    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7"} err="failed to get container status \"48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7\": rpc error: code = NotFound desc = could not find container \"48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7\": container with ID starting with 48b5550f82415361f86248c73ccacfbfbbf16c9cc224e614644f71e87d98c9d7 not found: ID does not exist"
Sep 17 16:20:13.485866 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:13.485845    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-5fd5fb8776-8jhrd"]
Sep 17 16:20:15.045060 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:15.045025    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" path="/var/lib/kubelet/pods/46fbf3be-1291-41f0-86fb-a4636eb991f7/volumes"
Sep 17 16:20:16.834231 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834197    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"]
Sep 17 16:20:16.834587 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834570    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="storage-initializer"
Sep 17 16:20:16.834587 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834581    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="storage-initializer"
Sep 17 16:20:16.834664 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834590    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kube-rbac-proxy"
Sep 17 16:20:16.834664 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834596    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kube-rbac-proxy"
Sep 17 16:20:16.834664 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834608    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container"
Sep 17 16:20:16.834664 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834613    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container"
Sep 17 16:20:16.834782 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834673    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kserve-container"
Sep 17 16:20:16.834782 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.834684    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="46fbf3be-1291-41f0-86fb-a4636eb991f7" containerName="kube-rbac-proxy"
Sep 17 16:20:16.839328 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.839308    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:16.842482 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.842461    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Sep 17 16:20:16.842787 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.842772    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:20:16.843529 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.843514    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:20:16.843609 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.843541    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-z8dwv\""
Sep 17 16:20:16.844062 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.844047    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:20:16.844264 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.844234    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Sep 17 16:20:16.844369 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.844297    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:20:16.855791 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.855769    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"]
Sep 17 16:20:16.944776 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.944721    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71e84127-eb05-4178-9187-d6170c183f6e-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:16.944887 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.944782    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71e84127-eb05-4178-9187-d6170c183f6e-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:16.944935 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.944880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mznlh\" (UniqueName: \"kubernetes.io/projected/71e84127-eb05-4178-9187-d6170c183f6e-kube-api-access-mznlh\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:16.944935 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.944907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:16.945027 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:16.944996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.045938 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.045909    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mznlh\" (UniqueName: \"kubernetes.io/projected/71e84127-eb05-4178-9187-d6170c183f6e-kube-api-access-mznlh\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.045938 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.045942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.046125 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.045979    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.046217 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.046196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71e84127-eb05-4178-9187-d6170c183f6e-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.046254 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.046231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71e84127-eb05-4178-9187-d6170c183f6e-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.046531 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.046514    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71e84127-eb05-4178-9187-d6170c183f6e-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.046674 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.046645    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.046775 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.046680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.048547 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.048530    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71e84127-eb05-4178-9187-d6170c183f6e-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.055110 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.055091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mznlh\" (UniqueName: \"kubernetes.io/projected/71e84127-eb05-4178-9187-d6170c183f6e-kube-api-access-mznlh\") pod \"isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.148900 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.148842    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:17.420980 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.420950    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"]
Sep 17 16:20:17.423587 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:20:17.423553    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e84127_eb05_4178_9187_d6170c183f6e.slice/crio-49cda2859ec5a2b5c99960b45297559a6ae47e444d42b54fe25afc777fcc8a85 WatchSource:0}: Error finding container 49cda2859ec5a2b5c99960b45297559a6ae47e444d42b54fe25afc777fcc8a85: Status 404 returned error can't find the container with id 49cda2859ec5a2b5c99960b45297559a6ae47e444d42b54fe25afc777fcc8a85
Sep 17 16:20:17.472973 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:17.472941    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerStarted","Data":"49cda2859ec5a2b5c99960b45297559a6ae47e444d42b54fe25afc777fcc8a85"}
Sep 17 16:20:18.479109 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:18.479070    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerStarted","Data":"1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd"}
Sep 17 16:20:21.495119 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:21.495091    2566 generic.go:356] "Generic (PLEG): container finished" podID="71e84127-eb05-4178-9187-d6170c183f6e" containerID="1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd" exitCode=0
Sep 17 16:20:21.495464 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:21.495178    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerDied","Data":"1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd"}
Sep 17 16:20:22.502970 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:22.502936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerStarted","Data":"de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c"}
Sep 17 16:20:22.503353 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:22.503020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerStarted","Data":"6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38"}
Sep 17 16:20:22.503353 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:22.503271    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:22.527778 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:22.527730    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podStartSLOduration=2.675404813 podStartE2EDuration="6.527714205s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.852309392s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:20:22.526437843 +0000 UTC m=+1571.963114623" watchObservedRunningTime="2026-09-17 16:20:22.527714205 +0000 UTC m=+1571.964390984"
Sep 17 16:20:23.508203 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:23.508150    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:23.509698 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:23.509666    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:20:24.513476 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:24.513435    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:20:29.518001 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:29.517968    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:20:29.518473 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:29.518447    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:20:39.519465 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:39.519421    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:20:49.518933 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:49.518890    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:20:59.519298 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:20:59.519256    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:21:09.519187 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:09.519088    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:21:19.519389 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:19.519339    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:21:29.518529 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:29.518486    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:21:37.044786 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:37.044758    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:21:46.940116 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:46.940080    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"]
Sep 17 16:21:46.940591 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:46.940421    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" containerID="cri-o://6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38" gracePeriod=30
Sep 17 16:21:46.940591 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:46.940503    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kube-rbac-proxy" containerID="cri-o://de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c" gracePeriod=30
Sep 17 16:21:47.039772 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:47.039730    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 16:21:47.924966 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:47.924932    2566 generic.go:356] "Generic (PLEG): container finished" podID="71e84127-eb05-4178-9187-d6170c183f6e" containerID="de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c" exitCode=2
Sep 17 16:21:47.925128 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:47.925000    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerDied","Data":"de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c"}
Sep 17 16:21:49.513903 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:49.513854    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.44:8643/healthz\": dial tcp 10.133.0.44:8643: connect: connection refused"
Sep 17 16:21:51.289996 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.289969    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:21:51.420922 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.420846    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mznlh\" (UniqueName: \"kubernetes.io/projected/71e84127-eb05-4178-9187-d6170c183f6e-kube-api-access-mznlh\") pod \"71e84127-eb05-4178-9187-d6170c183f6e\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") "
Sep 17 16:21:51.420922 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.420885    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-kube-rbac-proxy-sar-config\") pod \"71e84127-eb05-4178-9187-d6170c183f6e\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") "
Sep 17 16:21:51.421140 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.420975    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71e84127-eb05-4178-9187-d6170c183f6e-proxy-tls\") pod \"71e84127-eb05-4178-9187-d6170c183f6e\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") "
Sep 17 16:21:51.421140 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.420992    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-cabundle-cert\") pod \"71e84127-eb05-4178-9187-d6170c183f6e\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") "
Sep 17 16:21:51.421140 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.421025    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71e84127-eb05-4178-9187-d6170c183f6e-kserve-provision-location\") pod \"71e84127-eb05-4178-9187-d6170c183f6e\" (UID: \"71e84127-eb05-4178-9187-d6170c183f6e\") "
Sep 17 16:21:51.421383 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.421359    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "71e84127-eb05-4178-9187-d6170c183f6e" (UID: "71e84127-eb05-4178-9187-d6170c183f6e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:21:51.421440 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.421397    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e84127-eb05-4178-9187-d6170c183f6e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "71e84127-eb05-4178-9187-d6170c183f6e" (UID: "71e84127-eb05-4178-9187-d6170c183f6e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:21:51.421479 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.421429    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "71e84127-eb05-4178-9187-d6170c183f6e" (UID: "71e84127-eb05-4178-9187-d6170c183f6e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:21:51.422985 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.422962    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e84127-eb05-4178-9187-d6170c183f6e-kube-api-access-mznlh" (OuterVolumeSpecName: "kube-api-access-mznlh") pod "71e84127-eb05-4178-9187-d6170c183f6e" (UID: "71e84127-eb05-4178-9187-d6170c183f6e"). InnerVolumeSpecName "kube-api-access-mznlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:21:51.423071 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.422983    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71e84127-eb05-4178-9187-d6170c183f6e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "71e84127-eb05-4178-9187-d6170c183f6e" (UID: "71e84127-eb05-4178-9187-d6170c183f6e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:21:51.522030 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.521994    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:21:51.522030 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.522027    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71e84127-eb05-4178-9187-d6170c183f6e-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:21:51.522030 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.522038    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71e84127-eb05-4178-9187-d6170c183f6e-cabundle-cert\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:21:51.522273 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.522076    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71e84127-eb05-4178-9187-d6170c183f6e-kserve-provision-location\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:21:51.522273 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.522086    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mznlh\" (UniqueName: \"kubernetes.io/projected/71e84127-eb05-4178-9187-d6170c183f6e-kube-api-access-mznlh\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:21:51.945643 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.945604    2566 generic.go:356] "Generic (PLEG): container finished" podID="71e84127-eb05-4178-9187-d6170c183f6e" containerID="6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38" exitCode=0
Sep 17 16:21:51.945827 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.945661    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"
Sep 17 16:21:51.945827 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.945682    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerDied","Data":"6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38"}
Sep 17 16:21:51.945827 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.945741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg" event={"ID":"71e84127-eb05-4178-9187-d6170c183f6e","Type":"ContainerDied","Data":"49cda2859ec5a2b5c99960b45297559a6ae47e444d42b54fe25afc777fcc8a85"}
Sep 17 16:21:51.945827 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.945770    2566 scope.go:117] "RemoveContainer" containerID="de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c"
Sep 17 16:21:51.954934 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.954918    2566 scope.go:117] "RemoveContainer" containerID="6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38"
Sep 17 16:21:51.962330 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.962315    2566 scope.go:117] "RemoveContainer" containerID="1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd"
Sep 17 16:21:51.969417 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.969399    2566 scope.go:117] "RemoveContainer" containerID="de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c"
Sep 17 16:21:51.969690 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:21:51.969673    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c\": container with ID starting with de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c not found: ID does not exist" containerID="de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c"
Sep 17 16:21:51.969745 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.969698    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c"} err="failed to get container status \"de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c\": rpc error: code = NotFound desc = could not find container \"de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c\": container with ID starting with de822e3109e904f142e65d917f2aeb7b8c2a82d466f503ef83c19ff13e266c3c not found: ID does not exist"
Sep 17 16:21:51.969745 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.969713    2566 scope.go:117] "RemoveContainer" containerID="6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38"
Sep 17 16:21:51.969976 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:21:51.969959    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38\": container with ID starting with 6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38 not found: ID does not exist" containerID="6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38"
Sep 17 16:21:51.970024 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.969983    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38"} err="failed to get container status \"6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38\": rpc error: code = NotFound desc = could not find container \"6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38\": container with ID starting with 6f8bd0336d7a0df7f569937fe96688a289ac035735c63e6117deb9fb74d5ac38 not found: ID does not exist"
Sep 17 16:21:51.970024 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.969998    2566 scope.go:117] "RemoveContainer" containerID="1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd"
Sep 17 16:21:51.970263 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:21:51.970243    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd\": container with ID starting with 1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd not found: ID does not exist" containerID="1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd"
Sep 17 16:21:51.970333 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.970270    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd"} err="failed to get container status \"1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd\": rpc error: code = NotFound desc = could not find container \"1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd\": container with ID starting with 1fa61a03debf55e5ad9b91a0b39554b456dcc2efa692c096c2f7c8fb435f58fd not found: ID does not exist"
Sep 17 16:21:51.976541 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.976516    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"]
Sep 17 16:21:51.986186 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:51.986149    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7c98bbb8c9-8vhdg"]
Sep 17 16:21:53.044583 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:21:53.044551    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e84127-eb05-4178-9187-d6170c183f6e" path="/var/lib/kubelet/pods/71e84127-eb05-4178-9187-d6170c183f6e/volumes"
Sep 17 16:22:01.640188 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640142    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"]
Sep 17 16:22:01.640539 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640518    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container"
Sep 17 16:22:01.640539 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640530    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container"
Sep 17 16:22:01.640642 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640545    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kube-rbac-proxy"
Sep 17 16:22:01.640642 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640551    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kube-rbac-proxy"
Sep 17 16:22:01.640642 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640562    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="storage-initializer"
Sep 17 16:22:01.640642 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640569    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="storage-initializer"
Sep 17 16:22:01.640642 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640622    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kserve-container"
Sep 17 16:22:01.640642 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.640629    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e84127-eb05-4178-9187-d6170c183f6e" containerName="kube-rbac-proxy"
Sep 17 16:22:01.645307 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.645284    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.650341 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.650318    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-cipn-kube-rbac-proxy-sar-config\""
Sep 17 16:22:01.651683 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.651660    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:22:01.652189 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.651683    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-cipn-predictor-serving-cert\""
Sep 17 16:22:01.652348 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.652331    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:22:01.652711 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.652623    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-z8dwv\""
Sep 17 16:22:01.663745 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.663718    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"]
Sep 17 16:22:01.807247 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.807219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/883b2d04-899d-4318-a757-93e1b65b26b4-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.807403 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.807254    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/883b2d04-899d-4318-a757-93e1b65b26b4-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.807403 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.807290    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5vxx\" (UniqueName: \"kubernetes.io/projected/883b2d04-899d-4318-a757-93e1b65b26b4-kube-api-access-c5vxx\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.908339 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.908266    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/883b2d04-899d-4318-a757-93e1b65b26b4-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.908339 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.908306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5vxx\" (UniqueName: \"kubernetes.io/projected/883b2d04-899d-4318-a757-93e1b65b26b4-kube-api-access-c5vxx\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.908496 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.908379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/883b2d04-899d-4318-a757-93e1b65b26b4-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.909023 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.909004    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/883b2d04-899d-4318-a757-93e1b65b26b4-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.910695 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.910675    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/883b2d04-899d-4318-a757-93e1b65b26b4-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.919653 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.919630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5vxx\" (UniqueName: \"kubernetes.io/projected/883b2d04-899d-4318-a757-93e1b65b26b4-kube-api-access-c5vxx\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-559d8\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:01.956929 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:01.956904    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:02.232061 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:02.232034    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"]
Sep 17 16:22:02.234141 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:22:02.234113    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883b2d04_899d_4318_a757_93e1b65b26b4.slice/crio-ae88125f0b4ec8aae6800771493c72a69ab494ff517b23f93609c4f7cf9f4d40 WatchSource:0}: Error finding container ae88125f0b4ec8aae6800771493c72a69ab494ff517b23f93609c4f7cf9f4d40: Status 404 returned error can't find the container with id ae88125f0b4ec8aae6800771493c72a69ab494ff517b23f93609c4f7cf9f4d40
Sep 17 16:22:02.236263 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:02.236247    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 16:22:02.998557 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:02.998521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" event={"ID":"883b2d04-899d-4318-a757-93e1b65b26b4","Type":"ContainerStarted","Data":"ae88125f0b4ec8aae6800771493c72a69ab494ff517b23f93609c4f7cf9f4d40"}
Sep 17 16:22:04.004736 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:04.004704    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" event={"ID":"883b2d04-899d-4318-a757-93e1b65b26b4","Type":"ContainerStarted","Data":"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"}
Sep 17 16:22:04.005105 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:04.004765    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" event={"ID":"883b2d04-899d-4318-a757-93e1b65b26b4","Type":"ContainerStarted","Data":"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"}
Sep 17 16:22:04.005105 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:04.004797    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:04.030516 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:04.030475    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" podStartSLOduration=1.7427748360000002 podStartE2EDuration="3.030462147s" totalImagesPullingTime="1.287687311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:22:04.030404285 +0000 UTC m=+1673.467081075" watchObservedRunningTime="2026-09-17 16:22:04.030462147 +0000 UTC m=+1673.467138926"
Sep 17 16:22:05.009924 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:05.009887    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:05.011675 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:05.011649    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:22:12.022914 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:22:12.022881    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:23:56.615444 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:56.615411    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-raw-cipn-predictor-9774bf55d-559d8_883b2d04-899d-4318-a757-93e1b65b26b4/kserve-container/0.log"
Sep 17 16:23:56.878752 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:56.878664    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"]
Sep 17 16:23:56.879018 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:56.878927    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kserve-container" containerID="cri-o://0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e" gracePeriod=30
Sep 17 16:23:56.879018 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:56.878950    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kube-rbac-proxy" containerID="cri-o://f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4" gracePeriod=30
Sep 17 16:23:57.019361 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.019326    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.45:8643/healthz\": dial tcp 10.133.0.45:8643: connect: connection refused"
Sep 17 16:23:57.122045 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.122021    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:23:57.238776 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.238754    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/883b2d04-899d-4318-a757-93e1b65b26b4-kube-rbac-proxy-sar-config\") pod \"883b2d04-899d-4318-a757-93e1b65b26b4\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") "
Sep 17 16:23:57.238935 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.238862    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5vxx\" (UniqueName: \"kubernetes.io/projected/883b2d04-899d-4318-a757-93e1b65b26b4-kube-api-access-c5vxx\") pod \"883b2d04-899d-4318-a757-93e1b65b26b4\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") "
Sep 17 16:23:57.239002 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.238975    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/883b2d04-899d-4318-a757-93e1b65b26b4-proxy-tls\") pod \"883b2d04-899d-4318-a757-93e1b65b26b4\" (UID: \"883b2d04-899d-4318-a757-93e1b65b26b4\") "
Sep 17 16:23:57.239131 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.239109    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/883b2d04-899d-4318-a757-93e1b65b26b4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "883b2d04-899d-4318-a757-93e1b65b26b4" (UID: "883b2d04-899d-4318-a757-93e1b65b26b4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:23:57.239303 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.239281    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/883b2d04-899d-4318-a757-93e1b65b26b4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:23:57.240815 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.240788    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883b2d04-899d-4318-a757-93e1b65b26b4-kube-api-access-c5vxx" (OuterVolumeSpecName: "kube-api-access-c5vxx") pod "883b2d04-899d-4318-a757-93e1b65b26b4" (UID: "883b2d04-899d-4318-a757-93e1b65b26b4"). InnerVolumeSpecName "kube-api-access-c5vxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:23:57.240908 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.240875    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/883b2d04-899d-4318-a757-93e1b65b26b4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "883b2d04-899d-4318-a757-93e1b65b26b4" (UID: "883b2d04-899d-4318-a757-93e1b65b26b4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:23:57.340407 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.340372    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/883b2d04-899d-4318-a757-93e1b65b26b4-proxy-tls\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:23:57.340407 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.340404    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5vxx\" (UniqueName: \"kubernetes.io/projected/883b2d04-899d-4318-a757-93e1b65b26b4-kube-api-access-c5vxx\") on node \"ip-10-0-131-144.ec2.internal\" DevicePath \"\""
Sep 17 16:23:57.548135 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.548063    2566 generic.go:356] "Generic (PLEG): container finished" podID="883b2d04-899d-4318-a757-93e1b65b26b4" containerID="f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4" exitCode=2
Sep 17 16:23:57.548135 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.548084    2566 generic.go:356] "Generic (PLEG): container finished" podID="883b2d04-899d-4318-a757-93e1b65b26b4" containerID="0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e" exitCode=2
Sep 17 16:23:57.548135 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.548121    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"
Sep 17 16:23:57.548378 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.548143    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" event={"ID":"883b2d04-899d-4318-a757-93e1b65b26b4","Type":"ContainerDied","Data":"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"}
Sep 17 16:23:57.548378 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.548208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" event={"ID":"883b2d04-899d-4318-a757-93e1b65b26b4","Type":"ContainerDied","Data":"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"}
Sep 17 16:23:57.548378 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.548237    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8" event={"ID":"883b2d04-899d-4318-a757-93e1b65b26b4","Type":"ContainerDied","Data":"ae88125f0b4ec8aae6800771493c72a69ab494ff517b23f93609c4f7cf9f4d40"}
Sep 17 16:23:57.548378 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.548244    2566 scope.go:117] "RemoveContainer" containerID="f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"
Sep 17 16:23:57.557266 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.557247    2566 scope.go:117] "RemoveContainer" containerID="0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"
Sep 17 16:23:57.564860 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.564842    2566 scope.go:117] "RemoveContainer" containerID="f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"
Sep 17 16:23:57.565107 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:23:57.565090    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4\": container with ID starting with f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4 not found: ID does not exist" containerID="f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"
Sep 17 16:23:57.565169 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.565116    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"} err="failed to get container status \"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4\": rpc error: code = NotFound desc = could not find container \"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4\": container with ID starting with f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4 not found: ID does not exist"
Sep 17 16:23:57.565169 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.565131    2566 scope.go:117] "RemoveContainer" containerID="0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"
Sep 17 16:23:57.565489 ip-10-0-131-144 kubenswrapper[2566]: E0917 16:23:57.565465    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e\": container with ID starting with 0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e not found: ID does not exist" containerID="0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"
Sep 17 16:23:57.565563 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.565497    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"} err="failed to get container status \"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e\": rpc error: code = NotFound desc = could not find container \"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e\": container with ID starting with 0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e not found: ID does not exist"
Sep 17 16:23:57.565563 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.565513    2566 scope.go:117] "RemoveContainer" containerID="f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"
Sep 17 16:23:57.565862 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.565840    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4"} err="failed to get container status \"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4\": rpc error: code = NotFound desc = could not find container \"f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4\": container with ID starting with f044c94aeb718394b86f395ffccbb6f8e422caf0c3fe0e608d37e53280ca20d4 not found: ID does not exist"
Sep 17 16:23:57.565918 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.565865    2566 scope.go:117] "RemoveContainer" containerID="0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"
Sep 17 16:23:57.566148 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.566128    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e"} err="failed to get container status \"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e\": rpc error: code = NotFound desc = could not find container \"0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e\": container with ID starting with 0ba1390ed2305d94486600c23aeebfa3319ac1f36436e44112d0db201596646e not found: ID does not exist"
Sep 17 16:23:57.574412 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.574381    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"]
Sep 17 16:23:57.578506 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:57.578486    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-559d8"]
Sep 17 16:23:59.045144 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:23:59.045106    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" path="/var/lib/kubelet/pods/883b2d04-899d-4318-a757-93e1b65b26b4/volumes"
Sep 17 16:24:11.157524 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:11.157492    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:24:11.160702 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:11.160680    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:24:11.165556 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:11.165533    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:24:11.168315 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:11.168298    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:24:33.100729 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:33.100699    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d4xzc_9e29d81f-e0e8-43f7-b035-ef4bc9510105/global-pull-secret-syncer/0.log"
Sep 17 16:24:33.242259 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:33.242217    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n7ws2_bc51a31a-7fdd-4dc9-85d0-a484829f79bf/konnectivity-agent/0.log"
Sep 17 16:24:33.350996 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:33.350782    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-144.ec2.internal_fbe89b2e26a424e79e58e066d84f237c/haproxy/0.log"
Sep 17 16:24:36.875095 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:36.875064    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1cf02ae-3902-4438-9b81-6c950cd5f195/alertmanager/0.log"
Sep 17 16:24:36.908056 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:36.908032    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1cf02ae-3902-4438-9b81-6c950cd5f195/config-reloader/0.log"
Sep 17 16:24:36.937110 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:36.937085    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1cf02ae-3902-4438-9b81-6c950cd5f195/kube-rbac-proxy-web/0.log"
Sep 17 16:24:36.963656 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:36.963631    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1cf02ae-3902-4438-9b81-6c950cd5f195/kube-rbac-proxy/0.log"
Sep 17 16:24:37.000683 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.000653    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1cf02ae-3902-4438-9b81-6c950cd5f195/kube-rbac-proxy-metric/0.log"
Sep 17 16:24:37.035302 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.035283    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1cf02ae-3902-4438-9b81-6c950cd5f195/prom-label-proxy/0.log"
Sep 17 16:24:37.061859 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.061837    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1cf02ae-3902-4438-9b81-6c950cd5f195/init-config-reloader/0.log"
Sep 17 16:24:37.123310 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.123282    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-rs89x_8e0f92ca-79a8-429d-bdd5-e2924cc47b6e/cluster-monitoring-operator/0.log"
Sep 17 16:24:37.319833 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.319808    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-f58fr_8cbca560-0e32-447b-a0ee-5c2280afca4a/monitoring-plugin/0.log"
Sep 17 16:24:37.567500 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.567469    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wzd5l_d5947eed-4564-4108-8b06-33222fe68cc4/node-exporter/0.log"
Sep 17 16:24:37.596502 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.596434    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wzd5l_d5947eed-4564-4108-8b06-33222fe68cc4/kube-rbac-proxy/0.log"
Sep 17 16:24:37.629902 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.629884    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wzd5l_d5947eed-4564-4108-8b06-33222fe68cc4/init-textfile/0.log"
Sep 17 16:24:37.668013 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.667990    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pfqf6_2d2fd29c-0300-4d5b-8c09-36c7a29f3e93/kube-rbac-proxy-main/0.log"
Sep 17 16:24:37.702661 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.702625    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pfqf6_2d2fd29c-0300-4d5b-8c09-36c7a29f3e93/kube-rbac-proxy-self/0.log"
Sep 17 16:24:37.737706 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:37.737689    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pfqf6_2d2fd29c-0300-4d5b-8c09-36c7a29f3e93/openshift-state-metrics/0.log"
Sep 17 16:24:38.120858 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.120824    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-jj4gl_0b27a7d5-5dd2-4cbf-9abc-c5642495ae31/prometheus-operator-admission-webhook/0.log"
Sep 17 16:24:38.166295 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.166260    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58cc499c6f-2tknt_a30c2931-425e-4d1c-bfb8-f303d0b20faf/telemeter-client/0.log"
Sep 17 16:24:38.203050 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.203026    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58cc499c6f-2tknt_a30c2931-425e-4d1c-bfb8-f303d0b20faf/reload/0.log"
Sep 17 16:24:38.230779 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.230756    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58cc499c6f-2tknt_a30c2931-425e-4d1c-bfb8-f303d0b20faf/kube-rbac-proxy/0.log"
Sep 17 16:24:38.265240 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.265221    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9f5cb6d45-scmg9_f72c4aee-2ba9-479b-916a-f60b46960d41/thanos-query/0.log"
Sep 17 16:24:38.302185 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.302143    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9f5cb6d45-scmg9_f72c4aee-2ba9-479b-916a-f60b46960d41/kube-rbac-proxy-web/0.log"
Sep 17 16:24:38.329085 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.329061    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9f5cb6d45-scmg9_f72c4aee-2ba9-479b-916a-f60b46960d41/kube-rbac-proxy/0.log"
Sep 17 16:24:38.358177 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.358139    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9f5cb6d45-scmg9_f72c4aee-2ba9-479b-916a-f60b46960d41/prom-label-proxy/0.log"
Sep 17 16:24:38.391959 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.391903    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9f5cb6d45-scmg9_f72c4aee-2ba9-479b-916a-f60b46960d41/kube-rbac-proxy-rules/0.log"
Sep 17 16:24:38.431139 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:38.431110    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9f5cb6d45-scmg9_f72c4aee-2ba9-479b-916a-f60b46960d41/kube-rbac-proxy-metrics/0.log"
Sep 17 16:24:39.792849 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:39.792820    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-4vjtr_e76500bb-5ea7-4268-b949-aa1b8266e81f/networking-console-plugin/0.log"
Sep 17 16:24:40.300197 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:40.300152    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/2.log"
Sep 17 16:24:40.307935 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:40.307910    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2zp6f_6c520bd2-3078-4ef1-a685-70e71b9467af/console-operator/3.log"
Sep 17 16:24:40.808343 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:40.808311    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bbff7bb64-nzknl_2ea95922-956d-44ae-b4c4-7c0676eba8a9/console/0.log"
Sep 17 16:24:41.105549 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.105473    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"]
Sep 17 16:24:41.105858 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.105845    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kube-rbac-proxy"
Sep 17 16:24:41.105902 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.105861    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kube-rbac-proxy"
Sep 17 16:24:41.105902 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.105870    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kserve-container"
Sep 17 16:24:41.105902 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.105876    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kserve-container"
Sep 17 16:24:41.106001 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.105926    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kube-rbac-proxy"
Sep 17 16:24:41.106001 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.105934    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="883b2d04-899d-4318-a757-93e1b65b26b4" containerName="kserve-container"
Sep 17 16:24:41.109031 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.109014    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.112019 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.112000    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp8j5\"/\"kube-root-ca.crt\""
Sep 17 16:24:41.112195 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.112179    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp8j5\"/\"openshift-service-ca.crt\""
Sep 17 16:24:41.113066 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.113050    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vp8j5\"/\"default-dockercfg-lbwkw\""
Sep 17 16:24:41.124445 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.124427    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"]
Sep 17 16:24:41.194364 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.194336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfskc\" (UniqueName: \"kubernetes.io/projected/49ff109a-eb66-44fb-a892-fc43033841dc-kube-api-access-lfskc\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.194472 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.194378    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-lib-modules\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.194472 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.194405    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-proc\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.194570 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.194475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-podres\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.194570 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.194516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-sys\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.295739 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.295717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-sys\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.295856 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.295765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfskc\" (UniqueName: \"kubernetes.io/projected/49ff109a-eb66-44fb-a892-fc43033841dc-kube-api-access-lfskc\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.295856 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.295839    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-sys\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.295981 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.295880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-lib-modules\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.295981 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.295930    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-proc\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.295981 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.295957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-podres\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.296125 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.295992    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-proc\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.296125 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.296031    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-lib-modules\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.296125 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.296081    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/49ff109a-eb66-44fb-a892-fc43033841dc-podres\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.319053 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.319028    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfskc\" (UniqueName: \"kubernetes.io/projected/49ff109a-eb66-44fb-a892-fc43033841dc-kube-api-access-lfskc\") pod \"perf-node-gather-daemonset-rnlzq\" (UID: \"49ff109a-eb66-44fb-a892-fc43033841dc\") " pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.418829 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.418777    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:41.709851 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.709757    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"]
Sep 17 16:24:41.712934 ip-10-0-131-144 kubenswrapper[2566]: W0917 16:24:41.712911    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod49ff109a_eb66_44fb_a892_fc43033841dc.slice/crio-55c09ee7b796aff8cc2b4567b837d896e65ed93fd646a095cb47384252daf715 WatchSource:0}: Error finding container 55c09ee7b796aff8cc2b4567b837d896e65ed93fd646a095cb47384252daf715: Status 404 returned error can't find the container with id 55c09ee7b796aff8cc2b4567b837d896e65ed93fd646a095cb47384252daf715
Sep 17 16:24:41.760126 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:41.760099    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq" event={"ID":"49ff109a-eb66-44fb-a892-fc43033841dc","Type":"ContainerStarted","Data":"55c09ee7b796aff8cc2b4567b837d896e65ed93fd646a095cb47384252daf715"}
Sep 17 16:24:42.529699 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:42.529661    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6qjsh_0be3ee88-c394-4754-8569-fde1369da3e1/dns/0.log"
Sep 17 16:24:42.553169 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:42.553143    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6qjsh_0be3ee88-c394-4754-8569-fde1369da3e1/kube-rbac-proxy/0.log"
Sep 17 16:24:42.709403 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:42.709370    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pmbs8_71cbb99d-076e-430c-92a1-a135f7c133b8/dns-node-resolver/0.log"
Sep 17 16:24:42.765759 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:42.765723    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq" event={"ID":"49ff109a-eb66-44fb-a892-fc43033841dc","Type":"ContainerStarted","Data":"d0ba872731d05a2c6d189e11c64aa6a7721be971ae9b3591e67b7a8fbc85c482"}
Sep 17 16:24:42.765893 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:42.765818    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:42.784937 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:42.784855    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq" podStartSLOduration=1.784839589 podStartE2EDuration="1.784839589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:24:42.783286581 +0000 UTC m=+1832.219963360" watchObservedRunningTime="2026-09-17 16:24:42.784839589 +0000 UTC m=+1832.221516396"
Sep 17 16:24:43.246113 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:43.246084    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-c6965dbd6-kpr7x_edae5621-2635-4270-bdf1-c6900c53f3a7/registry/0.log"
Sep 17 16:24:43.381930 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:43.381895    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jj572_4c789c20-2a1f-4b93-8b12-d9c8144c53a7/node-ca/0.log"
Sep 17 16:24:44.567527 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:44.567495    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vrqvr_9153cbed-c147-4f00-8d05-7015cb79544e/serve-healthcheck-canary/0.log"
Sep 17 16:24:44.937949 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:44.937867    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-btkfl_ccb002d7-89bf-4dd2-8909-8701ab7755f6/insights-operator/0.log"
Sep 17 16:24:44.938572 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:44.938541    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-btkfl_ccb002d7-89bf-4dd2-8909-8701ab7755f6/insights-operator/1.log"
Sep 17 16:24:45.108821 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:45.108794    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ql9hb_d9162039-1eba-40eb-af07-47697bc0cc05/kube-rbac-proxy/0.log"
Sep 17 16:24:45.132201 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:45.132175    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ql9hb_d9162039-1eba-40eb-af07-47697bc0cc05/exporter/0.log"
Sep 17 16:24:45.157500 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:45.157471    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ql9hb_d9162039-1eba-40eb-af07-47697bc0cc05/extractor/0.log"
Sep 17 16:24:47.612179 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:47.612128    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-qsblw_0c963ef4-c5c2-4761-8abf-db8b9f21a460/manager/0.log"
Sep 17 16:24:47.918095 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:47.918019    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-2vnns_bd3e5381-7108-406a-b447-be83ea6dc841/seaweedfs/0.log"
Sep 17 16:24:48.780849 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:48.780823    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vp8j5/perf-node-gather-daemonset-rnlzq"
Sep 17 16:24:52.819597 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:52.819558    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-7564n_f02766b9-9dda-442e-8486-b9865cae794f/kube-storage-version-migrator-operator/1.log"
Sep 17 16:24:52.821356 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:52.821335    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-7564n_f02766b9-9dda-442e-8486-b9865cae794f/kube-storage-version-migrator-operator/0.log"
Sep 17 16:24:53.856543 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:53.856509    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6hslz_67289bb4-1805-4d07-8583-aae240420d82/kube-multus/0.log"
Sep 17 16:24:54.046630 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.046602    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bwzqr_a4244309-a935-44d0-9fe3-3edb7feb081f/kube-multus-additional-cni-plugins/0.log"
Sep 17 16:24:54.069295 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.069271    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bwzqr_a4244309-a935-44d0-9fe3-3edb7feb081f/egress-router-binary-copy/0.log"
Sep 17 16:24:54.093799 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.093736    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bwzqr_a4244309-a935-44d0-9fe3-3edb7feb081f/cni-plugins/0.log"
Sep 17 16:24:54.116176 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.116135    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bwzqr_a4244309-a935-44d0-9fe3-3edb7feb081f/bond-cni-plugin/0.log"
Sep 17 16:24:54.139950 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.139928    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bwzqr_a4244309-a935-44d0-9fe3-3edb7feb081f/routeoverride-cni/0.log"
Sep 17 16:24:54.164403 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.164384    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bwzqr_a4244309-a935-44d0-9fe3-3edb7feb081f/whereabouts-cni-bincopy/0.log"
Sep 17 16:24:54.192911 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.192889    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bwzqr_a4244309-a935-44d0-9fe3-3edb7feb081f/whereabouts-cni/0.log"
Sep 17 16:24:54.547369 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.547341    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vvjn8_26f8edd7-4332-49c1-8f85-00a63475c7ed/network-metrics-daemon/0.log"
Sep 17 16:24:54.571692 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:54.571667    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vvjn8_26f8edd7-4332-49c1-8f85-00a63475c7ed/kube-rbac-proxy/0.log"
Sep 17 16:24:55.422744 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.422709    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-controller/0.log"
Sep 17 16:24:55.446360 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.446337    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/1.log"
Sep 17 16:24:55.446360 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.446357    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovn-acl-logging/0.log"
Sep 17 16:24:55.469427 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.469399    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/kube-rbac-proxy-node/0.log"
Sep 17 16:24:55.494168 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.494127    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 16:24:55.514711 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.514689    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/northd/0.log"
Sep 17 16:24:55.537993 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.537972    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/nbdb/0.log"
Sep 17 16:24:55.560822 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.560801    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/sbdb/0.log"
Sep 17 16:24:55.615746 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:55.615722    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h25hc_228d91b2-8567-46ec-a469-c967cb61117c/ovnkube-controller/0.log"
Sep 17 16:24:57.256088 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:57.256061    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7cbc45fddc-zfb4v_bac97315-91c7-4dee-9c2a-3b05a65454e0/check-endpoints/0.log"
Sep 17 16:24:57.337315 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:57.337281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-lbkhx_5152d327-3905-4130-bc08-0e23cf39a6b2/network-check-target-container/0.log"
Sep 17 16:24:58.401044 ip-10-0-131-144 kubenswrapper[2566]: I0917 16:24:58.401012    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-lqrw5_20531a4e-3ef6-4648-b15e-540ffd02bcc8/iptables-alerter/0.log"