Sep 17 21:56:01.372516 ip-10-0-131-69 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 21:56:01.373021 ip-10-0-131-69 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 21:56:01.373145 ip-10-0-131-69 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 21:56:01.373697 ip-10-0-131-69 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 21:56:11.619389 ip-10-0-131-69 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 21:56:11.619405 ip-10-0-131-69 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7f1ea221e4824d3da89449c4209ef3aa --
Sep 17 21:57:47.864500 ip-10-0-131-69 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 21:57:50.982930 ip-10-0-131-69 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 21:57:50.982930 ip-10-0-131-69 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 21:57:50.982930 ip-10-0-131-69 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 21:57:50.982930 ip-10-0-131-69 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 21:57:50.982930 ip-10-0-131-69 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 21:57:50.983588 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.983012    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 21:57:50.986408 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986391    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:57:50.986408 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986408    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986412    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986416    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986418    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986422    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986425    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986427    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986430    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986433    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986435    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986438    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986441    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986450    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986453    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986457    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986460    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986462    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986465    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986468    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986470    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:57:50.986470 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986473    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986476    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986479    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986482    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986485    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986488    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986490    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986493    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986495    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986498    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986501    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986503    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986506    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986509    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986511    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986514    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986517    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986520    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986522    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:57:50.986940 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986524    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986527    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986530    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986532    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986535    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986537    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986540    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986542    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986547    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986550    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986553    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986557    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986560    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986563    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986566    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986569    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986572    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986575    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986578    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:57:50.987425 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986581    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986585    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986589    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986591    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986594    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986597    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986601    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986603    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986606    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986608    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986611    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986613    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986616    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986618    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986621    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986625    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986627    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986630    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986633    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986636    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:57:50.987888 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986638    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986641    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986644    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986646    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986649    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986651    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986654    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986656    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986660    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986662    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986665    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986668    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986670    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986673    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986675    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986678    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986681    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986684    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986686    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.986689    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:57:50.988383 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987394    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987401    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987404    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987407    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987410    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987412    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987415    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987418    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987421    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987424    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987426    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987430    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987432    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987436    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987438    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987441    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987443    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987446    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987448    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987451    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:57:50.988880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987455    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987457    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987460    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987463    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987465    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987470    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987474    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987477    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987480    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987483    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987486    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987489    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987493    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987497    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987499    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987502    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987504    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987507    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987510    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:57:50.989396 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987512    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987515    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987518    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987520    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987523    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987526    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987528    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987531    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987533    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987536    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987539    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987541    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987544    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987547    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987550    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987552    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987555    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987557    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987560    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:57:50.989855 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987564    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987567    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987570    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987573    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987576    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987578    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987581    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987583    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987586    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987588    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987591    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987593    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987596    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987598    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987601    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987603    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987606    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987609    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987611    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:57:50.990338 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987614    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987616    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987619    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987621    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987625    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987627    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987631    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987634    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987637    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987640    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987642    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987645    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987647    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987650    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987653    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987655    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987658    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987660    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987663    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987665    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:57:50.990789 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987668    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.987670    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988026    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988031    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988034    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988037    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988040    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988043    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988046    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988049    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988052    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988054    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988057    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988060    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988063    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988065    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988068    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988071    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988074    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988077    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:57:50.991300 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988081    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988085    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988088    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988091    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988094    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988098    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988102    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988105    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988108    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988110    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988113    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988116    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988118    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988121    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988124    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988127    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988130    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988132    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988135    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:57:50.991788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988137    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988140    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988143    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988145    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988148    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988151    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988153    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988156    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988158    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988161    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988163    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988166    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988169    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988172    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988175    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988177    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988180    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988182    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988185    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988187    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:57:50.992263 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988190    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988192    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988195    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988197    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988200    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988203    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988205    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988208    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988211    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988214    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988216    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988219    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988221    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988224    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988226    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988229    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988232    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988235    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988237    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988240    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:57:50.992788 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988242    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988261    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988264    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988267    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988269    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988272    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988275    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988277    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988280    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988283    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988285    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988288    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988291    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988293    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988296    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988299    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988301    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988304    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988307    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988309    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:57:50.993283 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988312    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.988316    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988388    2561 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988421    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988428    2561 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988432    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988436    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988439    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988444    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988457    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988461    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988464    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988468    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988471    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988474    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988478    2561 flags.go:64] FLAG: --cgroup-root=""
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988481    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988484    2561 flags.go:64] FLAG: --client-ca-file=""
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988487    2561 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988489    2561 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988494    2561 flags.go:64] FLAG: --cluster-domain=""
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988497    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988500    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 21:57:50.993761 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988503    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988506    2561 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988510    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988514    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988517    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988520    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988523    2561 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988526    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988529    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988532    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988534    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988538    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988541    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988544    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988547    2561 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988550    2561 flags.go:64] FLAG: --enable-server="true"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988553    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988558    2561 flags.go:64] FLAG: --event-burst="100"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988561    2561 flags.go:64] FLAG: --event-qps="50"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988565    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988568    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988571    2561 flags.go:64] FLAG: --eviction-hard=""
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988575    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988578    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988581    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 21:57:50.994377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988584    2561 flags.go:64] FLAG: --eviction-soft=""
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988586    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988589    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988592    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988595    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988598    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988601    2561 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988604    2561 flags.go:64] FLAG: --feature-gates=""
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988608    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988611    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988613    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988617    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988620    2561 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988622    2561 flags.go:64] FLAG: --help="false"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988625    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-131-69.ec2.internal"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988628    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988631    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988634    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988637    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988641    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988643    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988646    2561 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988649    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 21:57:50.995027 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988651    2561 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988654    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988658    2561 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988661    2561 flags.go:64] FLAG: --kube-reserved=""
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988664    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988667    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988669    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988672    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988675    2561 flags.go:64] FLAG: --lock-file=""
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988678    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988680    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988683    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988689    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988692    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988694    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988697    2561 flags.go:64] FLAG: --logging-format="text"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988700    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988703    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988706    2561 flags.go:64] FLAG: --manifest-url=""
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988708    2561 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988713    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988716    2561 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988720    2561 flags.go:64] FLAG: --max-pods="110"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988724    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988727    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 21:57:50.995914 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988729    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988732    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988735    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988738    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988740    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988747    2561 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988750    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988753    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988756    2561 flags.go:64] FLAG: --pod-cidr=""
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988759    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988765    2561 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988767    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988774    2561 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988777    2561 flags.go:64] FLAG: --port="10250"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988780    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988783    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d1c0be3b78e30030"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988786    2561 flags.go:64] FLAG: --qos-reserved=""
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988789    2561 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988791    2561 flags.go:64] FLAG: --register-node="true"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988794    2561 flags.go:64] FLAG: --register-with-taints=""
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988798    2561 flags.go:64] FLAG: --registry-burst="10"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988801    2561 flags.go:64] FLAG: --registry-qps="5"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988803    2561 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988806    2561 flags.go:64] FLAG: --reserved-memory=""
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988810    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 21:57:50.996570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988813    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988815    2561 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988818    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988821    2561 flags.go:64] FLAG: --runonce="false"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988824    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988828    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988831    2561 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988833    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988836    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988839    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988842    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988845    2561 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988848    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988851    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988854    2561 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988858    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988862    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988864    2561 flags.go:64] FLAG: --system-cgroups=""
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988868    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988873    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988877    2561 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988880    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988884    2561 flags.go:64] FLAG: --tls-min-version=""
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988887    2561 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988890    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 21:57:50.997221 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988893    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988896    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988899    2561 flags.go:64] FLAG: --v="2"
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988903    2561 flags.go:64] FLAG: --version="false"
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988907    2561 flags.go:64] FLAG: --vmodule=""
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988911    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.988914    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989049    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989053    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989057    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989060    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989063    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989066    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989069    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989072    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989075    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989077    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989080    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989083    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989085    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989088    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:57:50.997880 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989091    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989096    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989099    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989101    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989107    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989111    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989115    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989119    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989122    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989125    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989128    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989130    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989133    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989136    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989139    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989142    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989144    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989147    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989150    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:57:50.998428 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989152    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989155    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989158    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989161    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989163    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989166    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989168    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989171    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989174    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989177    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989179    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989182    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989185    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989187    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989191    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989194    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989197    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989201    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989203    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989206    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:57:50.998892 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989209    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989212    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989215    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989219    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989223    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989225    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989228    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989231    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989233    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989236    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989238    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989241    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989257    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989261    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989265    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989268    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989271    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989274    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989277    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:57:50.999404 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989280    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989282    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989285    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989287    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989290    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989293    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989295    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989299    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989302    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989305    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989309    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989312    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989314    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989317    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989320    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989323    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989326    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989328    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989331    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:57:50.999877 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989334    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989337    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989339    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989342    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989345    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989348    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989350    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.989353    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.989358    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.997306    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.997321    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.997342    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.997347    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997404    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997408    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:57:51.000367 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997411    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997415    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997419    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997423    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997426    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997429    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997432    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997435    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997438    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997440    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997443    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997446    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997450    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997454    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997457    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997460    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997462    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997465    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997468    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:57:51.000745 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997470    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997473    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997476    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997479    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997482    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997485    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997488    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997490    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997493    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997496    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997499    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997501    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997504    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997506    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997510    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997512    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997515    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997518    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997520    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997523    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:57:51.001208 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997526    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997528    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997531    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997533    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997536    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997539    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997541    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997544    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997546    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997549    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997552    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997555    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997559    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997562    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997565    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997567    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997570    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997573    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997575    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:57:51.001703 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997578    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997581    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997584    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997587    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997589    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997592    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997594    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997597    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997600    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997603    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997606    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997609    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997611    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997614    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997616    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997619    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997621    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997624    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997626    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:57:51.002171 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997629    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997631    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997634    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997637    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997639    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997642    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997645    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997648    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997650    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997653    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997656    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997658    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997661    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997664    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997667    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997669    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997672    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997674    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997677    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:57:51.002690 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997680    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.997685    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997790    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997795    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997798    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997801    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997804    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997807    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997809    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997812    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997815    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997817    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997819    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997822    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997826    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 21:57:51.003155 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997828    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997831    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997835    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997840    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997843    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997846    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997849    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997852    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997855    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997857    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997860    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997863    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997866    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997869    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997872    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997875    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997877    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997880    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997882    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 21:57:51.003569 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997885    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997888    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997891    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997894    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997897    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997899    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997902    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997905    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997907    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997911    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997914    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997917    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997920    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997923    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997926    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997929    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997932    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997935    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997937    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997940    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 21:57:51.004064 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997942    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997945    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997948    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997950    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997953    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997955    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997958    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997961    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997963    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997966    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997968    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997971    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997975    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997977    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997980    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997982    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997985    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997987    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997990    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 21:57:51.004696 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997993    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997995    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.997998    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998000    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998003    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998005    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998008    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998011    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998014    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998017    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998020    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998022    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998025    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998028    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998031    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998033    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998036    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998038    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998041    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998044    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 21:57:51.005159 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998046    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998049    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998051    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998054    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998056    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998059    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998062    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:50.998065    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.998070    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:50.998164    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.001096    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.001788    2561 server.go:1019] "Starting client certificate rotation"
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.001883    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 21:57:51.005706 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.001921    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 21:57:51.006049 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.003759    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 21:57:51.006049 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.005448    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 21:57:51.012801 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.008695    2561 log.go:25] "Validated CRI v1 runtime API"
Sep 17 21:57:51.012801 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.010068    2561 log.go:25] "Validated CRI v1 image API"
Sep 17 21:57:51.012801 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.011206    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 21:57:51.013393 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.013377    2561 fs.go:135] Filesystem UUIDs: map[32be3f62-74d2-422c-8829-e7979bc90f7e:/dev/nvme0n1p3 4df6ba54-5aa5-4a0c-b689-f97a029cd82f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 17 21:57:51.013433 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.013394    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 21:57:51.018846 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.018686    2561 manager.go:216] Machine: {Timestamp:2026-09-17 21:57:51.017687677 +0000 UTC m=+1.832063047 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100220 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27bca903c948b40df12f80d97858b8 SystemUUID:ec27bca9-03c9-48b4-0df1-2f80d97858b8 BootID:7f1ea221-e482-4d3d-a894-49c4209ef3aa Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ff:16:3b:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ff:16:3b:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:47:22:28:4a:43 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 21:57:51.018846 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.018832    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 21:57:51.019002 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.018935    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 21:57:51.020331 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.020303    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 21:57:51.020494 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.020333    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-69.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 21:57:51.021353 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.021341    2561 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 21:57:51.021425 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.021360    2561 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 21:57:51.021425 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.021371    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 21:57:51.021425 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.021386    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 21:57:51.021559 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.021471    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 21:57:51.025005 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.024991    2561 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 21:57:51.025139 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025128    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 21:57:51.025193 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025170    2561 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 21:57:51.025193 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025184    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 21:57:51.025310 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025199    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 21:57:51.025310 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025212    2561 kubelet.go:398] "Adding apiserver pod source"
Sep 17 21:57:51.025310 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025223    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 21:57:51.025589 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025566    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 21:57:51.025589 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.025588    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 21:57:51.029686 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.029668    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 21:57:51.031311 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.031289    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 21:57:51.031574 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.031560    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 21:57:51.031628 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.031596    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 21:57:51.032523 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032512    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 21:57:51.032576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032528    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 21:57:51.032576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032534    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 21:57:51.032576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032539    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 21:57:51.032576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032549    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 21:57:51.032576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032557    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 21:57:51.032576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032576    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032585    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.032580    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032595    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032625    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032636    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032641    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032665    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.032674    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 21:57:51.032775 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.032734    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-69.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 21:57:51.036837 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.036819    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-69.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 21:57:51.039617 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.039604    2561 server.go:1268] "Started kubelet"
Sep 17 21:57:51.039730 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.039664    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 21:57:51.039730 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.039676    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 21:57:51.039730 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.039722    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 21:57:51.040409 ip-10-0-131-69 systemd[1]: Started Kubernetes Kubelet.
Sep 17 21:57:51.040983 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.040627    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 21:57:51.041521 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.041366    2561 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 21:57:51.043756 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.043740    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 21:57:51.045975 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.045958    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 21:57:51.045975 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.045969    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 21:57:51.046136 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.046027    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.046432 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.046412    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 21:57:51.046512 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.046484    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 21:57:51.046512 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.046499    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 21:57:51.046940 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.046922    2561 factory.go:55] Registering systemd factory
Sep 17 21:57:51.046940 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.046941    2561 factory.go:223] Registration of the systemd container factory successfully
Sep 17 21:57:51.047127 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.047114    2561 factory.go:153] Registering CRI-O factory
Sep 17 21:57:51.047191 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.047131    2561 factory.go:223] Registration of the crio container factory successfully
Sep 17 21:57:51.047191 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.047179    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 21:57:51.047295 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.047196    2561 factory.go:103] Registering Raw factory
Sep 17 21:57:51.047295 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.047211    2561 manager.go:1196] Started watching for new ooms in manager
Sep 17 21:57:51.065519 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.065205    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-69.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 21:57:51.065519 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.065241    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 21:57:51.065519 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.065422    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bj2ld"
Sep 17 21:57:51.065519 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.065451    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bj2ld"
Sep 17 21:57:51.065790 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.065766    2561 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 21:57:51.065790 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.065781    2561 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 21:57:51.065958 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.065944    2561 manager.go:349] Starting recovery of all containers
Sep 17 21:57:51.076574 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.076559    2561 manager.go:354] Recovery completed
Sep 17 21:57:51.080803 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.080789    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:57:51.082751 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.082734    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:57:51.082813 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.082766    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:57:51.082813 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.082779    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:57:51.083309 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.083296    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 21:57:51.083371 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.083310    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 21:57:51.083371 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.083328    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:57:51.083436 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.083332    2561 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 21:57:51.084012 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.083999    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:57:51.084063 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.084022    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:57:51.084063 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.084034    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:57:51.085438 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.085425    2561 policy_none.go:49] "None policy: Start"
Sep 17 21:57:51.085481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.085443    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 21:57:51.085481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.085456    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 21:57:51.087457 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.087447    2561 policy_none.go:47] "Start"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.122725    2561 manager.go:337] "Starting Device Plugin manager"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.122758    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.122771    2561 server.go:87] "Starting device plugin registration server"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.123046    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.123078    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.123144    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.123287    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.123300    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.123912    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.123955    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.125376    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.126508    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.126554    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.126579    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.126622    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 21:57:51.138483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.128905    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:57:51.224171 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.224146    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:57:51.225324 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.225308    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:57:51.225418 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.225335    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:57:51.225418 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.225345    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:57:51.225418 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.225379    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.227463 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.227446    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal"]
Sep 17 21:57:51.227522 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.227513    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:57:51.228288 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.228263    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:57:51.228288 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.228286    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:57:51.228416 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.228295    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:57:51.230609 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.230593    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:57:51.230712 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.230696    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.230768 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.230742    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:57:51.231378 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.231364    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:57:51.231457 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.231389    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:57:51.231457 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.231399    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:57:51.231457 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.231367    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:57:51.231548 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.231462    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:57:51.231548 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.231471    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:57:51.233627 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.233585    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.233627 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.233622    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 21:57:51.233951 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.233933    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.234022 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.233960    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-69.ec2.internal\": node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.234650 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.234636    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 21:57:51.234720 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.234663    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 21:57:51.234720 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.234688    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeHasSufficientPID"
Sep 17 21:57:51.247122 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.247105    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.253846 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.253824    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-69.ec2.internal\" not found" node="ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.258397 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.258383    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-69.ec2.internal\" not found" node="ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.266525 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.266501    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7866f8af40529517a43d922924cc06dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal\" (UID: \"7866f8af40529517a43d922924cc06dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.266594 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.266530    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb7e1d05bdbb0cfb0bbd514a8ce678b1-config\") pod \"kube-apiserver-proxy-ip-10-0-131-69.ec2.internal\" (UID: \"cb7e1d05bdbb0cfb0bbd514a8ce678b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.266594 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.266558    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7866f8af40529517a43d922924cc06dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal\" (UID: \"7866f8af40529517a43d922924cc06dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.347979 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.347944    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.366693 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.366673    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7866f8af40529517a43d922924cc06dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal\" (UID: \"7866f8af40529517a43d922924cc06dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.366758 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.366696    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7866f8af40529517a43d922924cc06dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal\" (UID: \"7866f8af40529517a43d922924cc06dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.366758 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.366715    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb7e1d05bdbb0cfb0bbd514a8ce678b1-config\") pod \"kube-apiserver-proxy-ip-10-0-131-69.ec2.internal\" (UID: \"cb7e1d05bdbb0cfb0bbd514a8ce678b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.366758 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.366748    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb7e1d05bdbb0cfb0bbd514a8ce678b1-config\") pod \"kube-apiserver-proxy-ip-10-0-131-69.ec2.internal\" (UID: \"cb7e1d05bdbb0cfb0bbd514a8ce678b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.366854 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.366758    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7866f8af40529517a43d922924cc06dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal\" (UID: \"7866f8af40529517a43d922924cc06dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.366854 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.366776    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7866f8af40529517a43d922924cc06dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal\" (UID: \"7866f8af40529517a43d922924cc06dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.448020 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.447994    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.548756 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.548704    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.556876 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.556861    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.561768 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.561755    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:51.649065 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.649040    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.749579 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.749560    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.850032 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.849980    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:51.922427 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:51.921866    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:57:51.950176 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:51.950152    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:52.001462 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.001442    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 21:57:52.001962 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.001553    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 21:57:52.001962 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.001592    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 21:57:52.050511 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:52.050488    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:52.072182 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.072155    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 21:52:51 +0000 UTC" deadline="2028-02-17 17:32:50.351647315 +0000 UTC"
Sep 17 21:57:52.072182 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.072181    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12427h34m58.279469128s"
Sep 17 21:57:52.150641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:52.150583    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:52.199076 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:52.199037    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7866f8af40529517a43d922924cc06dc.slice/crio-deb74687732e2b19db12165a6dc8ae07a2c27ad4486cc6830a50e16913af2114 WatchSource:0}: Error finding container deb74687732e2b19db12165a6dc8ae07a2c27ad4486cc6830a50e16913af2114: Status 404 returned error can't find the container with id deb74687732e2b19db12165a6dc8ae07a2c27ad4486cc6830a50e16913af2114
Sep 17 21:57:52.199440 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:52.199422    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7e1d05bdbb0cfb0bbd514a8ce678b1.slice/crio-43ceb3ea62c5794c1d3514217b200a6ee0dc179a0b314f675e5769b9291f4c93 WatchSource:0}: Error finding container 43ceb3ea62c5794c1d3514217b200a6ee0dc179a0b314f675e5769b9291f4c93: Status 404 returned error can't find the container with id 43ceb3ea62c5794c1d3514217b200a6ee0dc179a0b314f675e5769b9291f4c93
Sep 17 21:57:52.205074 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.205061    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 21:57:52.251685 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:52.251664    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-69.ec2.internal\" not found"
Sep 17 21:57:52.295437 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.295419    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:57:52.346164 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.346146    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:52.355046 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.355031    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 21:57:52.355831 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.355818    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal"
Sep 17 21:57:52.364854 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.364840    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 21:57:52.449434 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:52.449392    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:57:53.026045 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.026007    2561 apiserver.go:52] "Watching apiserver"
Sep 17 21:57:53.032867 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.032841    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 21:57:53.033283 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.033261    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-tpvqh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6","openshift-cluster-node-tuning-operator/tuned-hx8mn","openshift-image-registry/node-ca-msktq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal","openshift-network-diagnostics/network-check-target-k7cfk","openshift-network-operator/iptables-alerter-cbzc2","openshift-ovn-kubernetes/ovnkube-node-24w8t","kube-system/global-pull-secret-syncer-djhj9","kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal","openshift-dns/node-resolver-6swmf","openshift-multus/multus-additional-cni-plugins-5qc2s","openshift-multus/multus-bxzz8","openshift-multus/network-metrics-daemon-xwm5p"]
Sep 17 21:57:53.035817 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.035796    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.038149 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.037990    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.038267 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.038169    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 21:57:53.038333 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.038311    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n68fq\""
Sep 17 21:57:53.038419 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.038332    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 21:57:53.038419 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.038398    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 21:57:53.039689 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.039660    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4nk4m\""
Sep 17 21:57:53.039841 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.039816    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 21:57:53.040503 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.040094    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 21:57:53.040503 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.040337    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.042924 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.042529    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rwtkk\""
Sep 17 21:57:53.042924 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.042677    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 21:57:53.042924 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.042803    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 21:57:53.043106 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.042935    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.044643 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.044626    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:53.044737 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.044708    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:57:53.046049 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.046030    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5nr5v\""
Sep 17 21:57:53.046143 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.046126    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 21:57:53.046199 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.046141    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 21:57:53.046465 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.046419    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 21:57:53.046588 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.046560    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 21:57:53.047257 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.047228    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:53.047338 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.047320    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:57:53.049752 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.049732    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.049837 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.049795    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:57:53.051994 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.051975    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.054376 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.054299    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 21:57:53.054635 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.054615    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 21:57:53.055328 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.055013    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 21:57:53.055328 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.055073    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 21:57:53.055328 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.055155    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-62twl\""
Sep 17 21:57:53.055518 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.055449    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.058536 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.058515    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.059458 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.059435    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 21:57:53.060060 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.060034    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fbxhn\""
Sep 17 21:57:53.060477 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.060459    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 21:57:53.060632 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.060613    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 21:57:53.060954 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.060938    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 21:57:53.062095 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.062069    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 21:57:53.062424 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.062403    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 21:57:53.062514 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.062441    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 21:57:53.062592 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.062555    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 21:57:53.062857 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.062695    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 21:57:53.062857 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.062728    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vsb7d\""
Sep 17 21:57:53.063543 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.063526    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 21:57:53.064358 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.064342    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.067064 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.067046    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 21:57:53.067217 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.067110    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kcphh\""
Sep 17 21:57:53.067527 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.067507    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.068593 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.068390    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 21:57:53.069847 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.069833    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 21:57:53.069968 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.069834    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 21:57:53.070058 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.069887    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ht4gl\""
Sep 17 21:57:53.075543 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075526    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-systemd\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.075742 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075721    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-host-slash\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.075832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075754    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536b96a-1b2e-4b48-96d6-128570aca0cc-tmp-dir\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.075832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075816    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysconfig\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.075939 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075835    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-host\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.075939 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075859    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qhvn\" (UniqueName: \"kubernetes.io/projected/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-kube-api-access-8qhvn\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.075939 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075917    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9811d2bc-c727-42e3-a514-02ca5a24be2f-kubelet-config\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.075939 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075936    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbnx8\" (UniqueName: \"kubernetes.io/projected/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-kube-api-access-gbnx8\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.076108 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.075957    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-serviceca\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.076108 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076022    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-run\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076108 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-sys\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076108 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076089    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fe371e6-2152-4ec7-8751-5b6d268b1e01-tmp\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076300 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-var-lib-kubelet\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076300 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076175    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:53.076300 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076192    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-host\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.076300 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076215    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:53.076300 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076239    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-567vm\" (UniqueName: \"kubernetes.io/projected/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-kube-api-access-567vm\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:53.076300 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076290    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-modprobe-d\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076311    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysctl-d\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-iptables-alerter-script\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.076576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076411    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-tuned\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076437    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9811d2bc-c727-42e3-a514-02ca5a24be2f-dbus\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.076576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076468    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrkq8\" (UniqueName: \"kubernetes.io/projected/e536b96a-1b2e-4b48-96d6-128570aca0cc-kube-api-access-xrkq8\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.076576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076492    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysctl-conf\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076576 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076538    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s829v\" (UniqueName: \"kubernetes.io/projected/5fe371e6-2152-4ec7-8751-5b6d268b1e01-kube-api-access-s829v\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076858 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076588    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.076858 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076621    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e536b96a-1b2e-4b48-96d6-128570aca0cc-hosts-file\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.076858 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076648    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-kubernetes\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.076858 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.076684    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-lib-modules\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.087780 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.087758    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zdnkw"
Sep 17 21:57:53.095963 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.095944    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zdnkw"
Sep 17 21:57:53.130514 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.130472    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal" event={"ID":"7866f8af40529517a43d922924cc06dc","Type":"ContainerStarted","Data":"deb74687732e2b19db12165a6dc8ae07a2c27ad4486cc6830a50e16913af2114"}
Sep 17 21:57:53.131703 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.131679    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal" event={"ID":"cb7e1d05bdbb0cfb0bbd514a8ce678b1","Type":"ContainerStarted","Data":"43ceb3ea62c5794c1d3514217b200a6ee0dc179a0b314f675e5769b9291f4c93"}
Sep 17 21:57:53.147136 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.147116    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 21:57:53.177759 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177734    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-systemd\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.177880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177769    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-host-slash\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.177880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177796    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536b96a-1b2e-4b48-96d6-128570aca0cc-tmp-dir\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.177880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysconfig\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.177880 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177866    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-host-slash\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177865    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-systemd\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177896    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-host\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177929    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysconfig\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177983    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9811d2bc-c727-42e3-a514-02ca5a24be2f-kubelet-config\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.177988    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-host\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178011    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbnx8\" (UniqueName: \"kubernetes.io/projected/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-kube-api-access-gbnx8\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-run-netns\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178052    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-etc-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.178073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/67602f97-3cbe-4453-aac5-114689901c4a-ovn-node-metrics-cert\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178079    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9811d2bc-c727-42e3-a514-02ca5a24be2f-kubelet-config\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178090    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-etc-kubernetes\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536b96a-1b2e-4b48-96d6-128570aca0cc-tmp-dir\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178124    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178217    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-registration-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178265    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-serviceca\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178292    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-run\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-cni-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178338    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-cni-binary-copy\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178373    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-run\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178377    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-kubelet\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178403    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbd374c7-122a-4732-808f-9777edcf8fa9-konnectivity-ca\") pod \"konnectivity-agent-tpvqh\" (UID: \"dbd374c7-122a-4732-808f-9777edcf8fa9\") " pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178432    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178459    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-567vm\" (UniqueName: \"kubernetes.io/projected/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-kube-api-access-567vm\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:53.178498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178485    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-modprobe-d\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.179057 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-modprobe-d\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.179057 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/776e2517-6042-4c45-b5ce-267e540962ad-cni-binary-copy\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.179057 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178677    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-hostroot\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.179057 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178702    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4vn8\" (UniqueName: \"kubernetes.io/projected/20010908-1f7c-4c73-98aa-8598bfced30d-kube-api-access-n4vn8\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.179057 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178709    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-serviceca\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.179057 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.178731    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-iptables-alerter-script\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179094    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179136    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-tuned\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-kubelet\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179182    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-systemd-units\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179204    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-system-cni-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179228    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-device-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179266    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-sys-fs\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-iptables-alerter-script\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179290    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-cnibin\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179340    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9811d2bc-c727-42e3-a514-02ca5a24be2f-dbus\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysctl-conf\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.179414 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179396    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s829v\" (UniqueName: \"kubernetes.io/projected/5fe371e6-2152-4ec7-8751-5b6d268b1e01-kube-api-access-s829v\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179422    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-conf-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179448    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179477    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-run-ovn-kubernetes\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179502    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtcwq\" (UniqueName: \"kubernetes.io/projected/67602f97-3cbe-4453-aac5-114689901c4a-kube-api-access-gtcwq\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179506    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9811d2bc-c727-42e3-a514-02ca5a24be2f-dbus\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179526    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-cnibin\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179542    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysctl-conf\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179549    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbd374c7-122a-4732-808f-9777edcf8fa9-agent-certs\") pod \"konnectivity-agent-tpvqh\" (UID: \"dbd374c7-122a-4732-808f-9777edcf8fa9\") " pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.179582    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179596    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-slash\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.179656    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret podName:9811d2bc-c727-42e3-a514-02ca5a24be2f nodeName:}" failed. No retries permitted until 2026-09-17 21:57:53.679637654 +0000 UTC m=+4.494013028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret") pod "global-pull-secret-syncer-djhj9" (UID: "9811d2bc-c727-42e3-a514-02ca5a24be2f") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179685    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-ovn\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179722    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-log-socket\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179760    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-cni-bin\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179789    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-socket-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179842    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qhvn\" (UniqueName: \"kubernetes.io/projected/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-kube-api-access-8qhvn\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.179928 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179866    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-ovnkube-config\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/776e2517-6042-4c45-b5ce-267e540962ad-multus-daemon-config\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179914    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179943    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-sys\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179967    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fe371e6-2152-4ec7-8751-5b6d268b1e01-tmp\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.179990    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-var-lib-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180005    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-sys\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180012    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-k8s-cni-cncf-io\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180065    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-cni-bin\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180093    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180120    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-systemd\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180139    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-socket-dir-parent\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180161    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-etc-selinux\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180185    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-system-cni-dir\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180208    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48zbh\" (UniqueName: \"kubernetes.io/projected/f10417bd-9a32-43ea-be80-6a2b4a414203-kube-api-access-48zbh\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180237    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-var-lib-kubelet\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180276    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-host\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.180764 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180304    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysctl-d\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180329    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-var-lib-kubelet\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-host\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180364    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-cni-netd\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180390    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-cni-multus\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180418    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.180428    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180458    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180462    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-sysctl-d\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.180495    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs podName:2e8cccc3-163f-4c1e-af6e-6390d5a2a694 nodeName:}" failed. No retries permitted until 2026-09-17 21:57:53.680478261 +0000 UTC m=+4.494853631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs") pod "network-metrics-daemon-xwm5p" (UID: "2e8cccc3-163f-4c1e-af6e-6390d5a2a694") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180519    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-ovnkube-script-lib\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180553    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-netns\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180582    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrkq8\" (UniqueName: \"kubernetes.io/projected/e536b96a-1b2e-4b48-96d6-128570aca0cc-kube-api-access-xrkq8\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180608    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-multus-certs\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180631    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p4vk\" (UniqueName: \"kubernetes.io/projected/776e2517-6042-4c45-b5ce-267e540962ad-kube-api-access-9p4vk\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180750    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e536b96a-1b2e-4b48-96d6-128570aca0cc-hosts-file\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.181404 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180791    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-kubernetes\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.181969 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180814    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-lib-modules\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.181969 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180840    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-node-log\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.181969 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180861    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-kubernetes\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.181969 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180867    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.181969 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180885    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e536b96a-1b2e-4b48-96d6-128570aca0cc-hosts-file\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.181969 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180893    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-env-overrides\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.181969 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.180951    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fe371e6-2152-4ec7-8751-5b6d268b1e01-lib-modules\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.183049 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.183024    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fe371e6-2152-4ec7-8751-5b6d268b1e01-tmp\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.183132 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.183053    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5fe371e6-2152-4ec7-8751-5b6d268b1e01-etc-tuned\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.187641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.187511    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:57:53.187641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.187534    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:57:53.187641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.187547    2561 projected.go:196] Error preparing data for projected volume kube-api-access-k6mm4 for pod openshift-network-diagnostics/network-check-target-k7cfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:53.187641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.187610    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4 podName:b3a32913-e0bd-40ff-b8c9-e80a8a7adc58 nodeName:}" failed. No retries permitted until 2026-09-17 21:57:53.687592214 +0000 UTC m=+4.501967571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k6mm4" (UniqueName: "kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4") pod "network-check-target-k7cfk" (UID: "b3a32913-e0bd-40ff-b8c9-e80a8a7adc58") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:53.190230 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.190196    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbnx8\" (UniqueName: \"kubernetes.io/projected/aa8136e8-83ac-4e34-ae35-c6c11d62f18f-kube-api-access-gbnx8\") pod \"iptables-alerter-cbzc2\" (UID: \"aa8136e8-83ac-4e34-ae35-c6c11d62f18f\") " pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.191087 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.190995    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-567vm\" (UniqueName: \"kubernetes.io/projected/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-kube-api-access-567vm\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:53.191168 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.191091    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s829v\" (UniqueName: \"kubernetes.io/projected/5fe371e6-2152-4ec7-8751-5b6d268b1e01-kube-api-access-s829v\") pod \"tuned-hx8mn\" (UID: \"5fe371e6-2152-4ec7-8751-5b6d268b1e01\") " pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.191260 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.191223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrkq8\" (UniqueName: \"kubernetes.io/projected/e536b96a-1b2e-4b48-96d6-128570aca0cc-kube-api-access-xrkq8\") pod \"node-resolver-6swmf\" (UID: \"e536b96a-1b2e-4b48-96d6-128570aca0cc\") " pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.191599 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.191578    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qhvn\" (UniqueName: \"kubernetes.io/projected/87535f94-4b13-4fb8-a307-eb2fdf5ec1d0-kube-api-access-8qhvn\") pod \"node-ca-msktq\" (UID: \"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0\") " pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.282167 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282085    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-socket-dir-parent\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.282167 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-etc-selinux\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.282167 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282151    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-system-cni-dir\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282178    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48zbh\" (UniqueName: \"kubernetes.io/projected/f10417bd-9a32-43ea-be80-6a2b4a414203-kube-api-access-48zbh\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282218    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-cni-netd\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-socket-dir-parent\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282235    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-etc-selinux\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-cni-multus\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-system-cni-dir\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282318    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-cni-netd\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282324    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282360    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-cni-multus\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282375    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-ovnkube-script-lib\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-netns\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282405    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.282443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282446    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-netns\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-multus-certs\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282497    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p4vk\" (UniqueName: \"kubernetes.io/projected/776e2517-6042-4c45-b5ce-267e540962ad-kube-api-access-9p4vk\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282521    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-node-log\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-multus-certs\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282547    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-env-overrides\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282575    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-node-log\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-run-netns\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282636    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-etc-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/67602f97-3cbe-4453-aac5-114689901c4a-ovn-node-metrics-cert\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282688    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-etc-kubernetes\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282710    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282735    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-registration-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282760    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-cni-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282785    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-cni-binary-copy\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282793    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283077 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282810    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-kubelet\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282834    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbd374c7-122a-4732-808f-9777edcf8fa9-konnectivity-ca\") pod \"konnectivity-agent-tpvqh\" (UID: \"dbd374c7-122a-4732-808f-9777edcf8fa9\") " pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282839    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-etc-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282883    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/776e2517-6042-4c45-b5ce-267e540962ad-cni-binary-copy\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282887    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282907    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-hostroot\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282758    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-run-netns\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282932    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4vn8\" (UniqueName: \"kubernetes.io/projected/20010908-1f7c-4c73-98aa-8598bfced30d-kube-api-access-n4vn8\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282958    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-kubelet\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282962    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-etc-kubernetes\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282982    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-systemd-units\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282986    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-ovnkube-script-lib\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283005    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-system-cni-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283008    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-registration-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.282916    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-env-overrides\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283027    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-device-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283050    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-sys-fs\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.283885 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283071    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-cnibin\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283097    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-conf-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283131    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-run-ovn-kubernetes\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283155    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcwq\" (UniqueName: \"kubernetes.io/projected/67602f97-3cbe-4453-aac5-114689901c4a-kube-api-access-gtcwq\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283179    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-cnibin\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbd374c7-122a-4732-808f-9777edcf8fa9-agent-certs\") pod \"konnectivity-agent-tpvqh\" (UID: \"dbd374c7-122a-4732-808f-9777edcf8fa9\") " pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283212    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283229    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-slash\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-ovn\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283284    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-hostroot\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283292    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-kubelet\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-log-socket\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283335    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-log-socket\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283340    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-cni-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283341    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbd374c7-122a-4732-808f-9777edcf8fa9-konnectivity-ca\") pod \"konnectivity-agent-tpvqh\" (UID: \"dbd374c7-122a-4732-808f-9777edcf8fa9\") " pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283376    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-multus-conf-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283416    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-run-ovn-kubernetes\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283430    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-sys-fs\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.284710 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283432    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-device-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283449    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-cnibin\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283460    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-slash\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283480    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-systemd-units\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283476    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-cni-binary-copy\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283501    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-cnibin\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283533    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-cni-bin\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283560    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-kubelet\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283534    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-ovn\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283560    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-socket-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283615    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-system-cni-dir\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-host-cni-bin\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283678    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-ovnkube-config\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283687    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/20010908-1f7c-4c73-98aa-8598bfced30d-socket-dir\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283719    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/776e2517-6042-4c45-b5ce-267e540962ad-multus-daemon-config\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283780    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283811    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/776e2517-6042-4c45-b5ce-267e540962ad-cni-binary-copy\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283887    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-var-lib-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.285567 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283926    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-k8s-cni-cncf-io\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283952    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-cni-bin\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283984    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f10417bd-9a32-43ea-be80-6a2b4a414203-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.283991    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-systemd\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284038    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-var-lib-cni-bin\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284062    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-var-lib-openvswitch\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284101    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/776e2517-6042-4c45-b5ce-267e540962ad-host-run-k8s-cni-cncf-io\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284131    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/67602f97-3cbe-4453-aac5-114689901c4a-ovnkube-config\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284135    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/67602f97-3cbe-4453-aac5-114689901c4a-run-systemd\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284204    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/776e2517-6042-4c45-b5ce-267e540962ad-multus-daemon-config\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.284513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f10417bd-9a32-43ea-be80-6a2b4a414203-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.285625    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/67602f97-3cbe-4453-aac5-114689901c4a-ovn-node-metrics-cert\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.286208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.286034    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbd374c7-122a-4732-808f-9777edcf8fa9-agent-certs\") pod \"konnectivity-agent-tpvqh\" (UID: \"dbd374c7-122a-4732-808f-9777edcf8fa9\") " pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.290188 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.290164    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48zbh\" (UniqueName: \"kubernetes.io/projected/f10417bd-9a32-43ea-be80-6a2b4a414203-kube-api-access-48zbh\") pod \"multus-additional-cni-plugins-5qc2s\" (UID: \"f10417bd-9a32-43ea-be80-6a2b4a414203\") " pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.290419 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.290398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4vn8\" (UniqueName: \"kubernetes.io/projected/20010908-1f7c-4c73-98aa-8598bfced30d-kube-api-access-n4vn8\") pod \"aws-ebs-csi-driver-node-r84c6\" (UID: \"20010908-1f7c-4c73-98aa-8598bfced30d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.290683 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.290664    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p4vk\" (UniqueName: \"kubernetes.io/projected/776e2517-6042-4c45-b5ce-267e540962ad-kube-api-access-9p4vk\") pod \"multus-bxzz8\" (UID: \"776e2517-6042-4c45-b5ce-267e540962ad\") " pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.291234 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.291217    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtcwq\" (UniqueName: \"kubernetes.io/projected/67602f97-3cbe-4453-aac5-114689901c4a-kube-api-access-gtcwq\") pod \"ovnkube-node-24w8t\" (UID: \"67602f97-3cbe-4453-aac5-114689901c4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.352260 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.352223    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cbzc2"
Sep 17 21:57:53.361803 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.361783    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:57:53.362829 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.362808    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6swmf"
Sep 17 21:57:53.374220 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.374202    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hx8mn"
Sep 17 21:57:53.381869 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.381849    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-msktq"
Sep 17 21:57:53.388500 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.388480    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bxzz8"
Sep 17 21:57:53.396086 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.396066    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6"
Sep 17 21:57:53.403671 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.403652    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:57:53.410328 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.410305    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:57:53.415819 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.415804    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5qc2s"
Sep 17 21:57:53.685893 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.685820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:53.685893 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.685876    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:53.686111 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.685941    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:53.686111 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.685996    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret podName:9811d2bc-c727-42e3-a514-02ca5a24be2f nodeName:}" failed. No retries permitted until 2026-09-17 21:57:54.685980686 +0000 UTC m=+5.500356046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret") pod "global-pull-secret-syncer-djhj9" (UID: "9811d2bc-c727-42e3-a514-02ca5a24be2f") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:53.686111 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.686021    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:53.686111 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.686079    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs podName:2e8cccc3-163f-4c1e-af6e-6390d5a2a694 nodeName:}" failed. No retries permitted until 2026-09-17 21:57:54.6860628 +0000 UTC m=+5.500438169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs") pod "network-metrics-daemon-xwm5p" (UID: "2e8cccc3-163f-4c1e-af6e-6390d5a2a694") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:53.786179 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:53.786149    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:53.786341 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.786313    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:57:53.786341 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.786335    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:57:53.786421 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.786348    2561 projected.go:196] Error preparing data for projected volume kube-api-access-k6mm4 for pod openshift-network-diagnostics/network-check-target-k7cfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:53.786421 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:53.786408    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4 podName:b3a32913-e0bd-40ff-b8c9-e80a8a7adc58 nodeName:}" failed. No retries permitted until 2026-09-17 21:57:54.786391364 +0000 UTC m=+5.600766740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6mm4" (UniqueName: "kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4") pod "network-check-target-k7cfk" (UID: "b3a32913-e0bd-40ff-b8c9-e80a8a7adc58") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:54.084514 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:54.084488    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe371e6_2152_4ec7_8751_5b6d268b1e01.slice/crio-de54b21f7124b7b2443a7fb679a1674f841ddb9485cc94076b8426e0aa07cb81 WatchSource:0}: Error finding container de54b21f7124b7b2443a7fb679a1674f841ddb9485cc94076b8426e0aa07cb81: Status 404 returned error can't find the container with id de54b21f7124b7b2443a7fb679a1674f841ddb9485cc94076b8426e0aa07cb81
Sep 17 21:57:54.086174 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:54.086135    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67602f97_3cbe_4453_aac5_114689901c4a.slice/crio-79eef84dd333a5af36cf524a1e9d513655af20ed89afd8e23df1e25b29e27ad1 WatchSource:0}: Error finding container 79eef84dd333a5af36cf524a1e9d513655af20ed89afd8e23df1e25b29e27ad1: Status 404 returned error can't find the container with id 79eef84dd333a5af36cf524a1e9d513655af20ed89afd8e23df1e25b29e27ad1
Sep 17 21:57:54.090200 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:54.090177    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10417bd_9a32_43ea_be80_6a2b4a414203.slice/crio-bd2945f6c65a02640530c776401cc8bd39dca1a6bf23f23e93140910f95c8ed7 WatchSource:0}: Error finding container bd2945f6c65a02640530c776401cc8bd39dca1a6bf23f23e93140910f95c8ed7: Status 404 returned error can't find the container with id bd2945f6c65a02640530c776401cc8bd39dca1a6bf23f23e93140910f95c8ed7
Sep 17 21:57:54.091600 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:54.091521    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode536b96a_1b2e_4b48_96d6_128570aca0cc.slice/crio-dbae92e8b9339c7eb014b60069a3d11ecbd29db762da2fbda85d3aec85a9b13d WatchSource:0}: Error finding container dbae92e8b9339c7eb014b60069a3d11ecbd29db762da2fbda85d3aec85a9b13d: Status 404 returned error can't find the container with id dbae92e8b9339c7eb014b60069a3d11ecbd29db762da2fbda85d3aec85a9b13d
Sep 17 21:57:54.092734 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:54.092715    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776e2517_6042_4c45_b5ce_267e540962ad.slice/crio-51f37ddde4b7c710126a5402c71f58c571159d662acaa8aa7161ed41265f42ba WatchSource:0}: Error finding container 51f37ddde4b7c710126a5402c71f58c571159d662acaa8aa7161ed41265f42ba: Status 404 returned error can't find the container with id 51f37ddde4b7c710126a5402c71f58c571159d662acaa8aa7161ed41265f42ba
Sep 17 21:57:54.093803 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:54.093782    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8136e8_83ac_4e34_ae35_c6c11d62f18f.slice/crio-2a18396d49a56d826fa68e2b6de8aefab51b13afcaf59f7ecd84a1e29aca7bb2 WatchSource:0}: Error finding container 2a18396d49a56d826fa68e2b6de8aefab51b13afcaf59f7ecd84a1e29aca7bb2: Status 404 returned error can't find the container with id 2a18396d49a56d826fa68e2b6de8aefab51b13afcaf59f7ecd84a1e29aca7bb2
Sep 17 21:57:54.096968 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:57:54.096473    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd374c7_122a_4732_808f_9777edcf8fa9.slice/crio-bddaa60eaf33d2ca70972cc05af9f5a89f3bb448db1d0dbe4a1a1f1a7146babd WatchSource:0}: Error finding container bddaa60eaf33d2ca70972cc05af9f5a89f3bb448db1d0dbe4a1a1f1a7146babd: Status 404 returned error can't find the container with id bddaa60eaf33d2ca70972cc05af9f5a89f3bb448db1d0dbe4a1a1f1a7146babd
Sep 17 21:57:54.099145 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.098894    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 21:52:53 +0000 UTC" deadline="2028-06-15 12:00:28.01172174 +0000 UTC"
Sep 17 21:57:54.099145 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.098919    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15278h2m33.912806738s"
Sep 17 21:57:54.133784 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.133672    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-msktq" event={"ID":"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0","Type":"ContainerStarted","Data":"f9ad675aa5a9d5a99b76a42048f05cf034e6305c405291151c213dfd62df7eb9"}
Sep 17 21:57:54.134767 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.134747    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6" event={"ID":"20010908-1f7c-4c73-98aa-8598bfced30d","Type":"ContainerStarted","Data":"c6307cd79984d8b9a7bb816663c0c7da8fb401514e6eccc5c2f56c6e9ba8f47b"}
Sep 17 21:57:54.135800 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.135781    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tpvqh" event={"ID":"dbd374c7-122a-4732-808f-9777edcf8fa9","Type":"ContainerStarted","Data":"bddaa60eaf33d2ca70972cc05af9f5a89f3bb448db1d0dbe4a1a1f1a7146babd"}
Sep 17 21:57:54.139212 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.139192    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6swmf" event={"ID":"e536b96a-1b2e-4b48-96d6-128570aca0cc","Type":"ContainerStarted","Data":"dbae92e8b9339c7eb014b60069a3d11ecbd29db762da2fbda85d3aec85a9b13d"}
Sep 17 21:57:54.141575 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.141556    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"79eef84dd333a5af36cf524a1e9d513655af20ed89afd8e23df1e25b29e27ad1"}
Sep 17 21:57:54.142717 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.142696    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cbzc2" event={"ID":"aa8136e8-83ac-4e34-ae35-c6c11d62f18f","Type":"ContainerStarted","Data":"2a18396d49a56d826fa68e2b6de8aefab51b13afcaf59f7ecd84a1e29aca7bb2"}
Sep 17 21:57:54.144068 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.144046    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bxzz8" event={"ID":"776e2517-6042-4c45-b5ce-267e540962ad","Type":"ContainerStarted","Data":"51f37ddde4b7c710126a5402c71f58c571159d662acaa8aa7161ed41265f42ba"}
Sep 17 21:57:54.145093 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.145074    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerStarted","Data":"bd2945f6c65a02640530c776401cc8bd39dca1a6bf23f23e93140910f95c8ed7"}
Sep 17 21:57:54.146547 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.146525    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hx8mn" event={"ID":"5fe371e6-2152-4ec7-8751-5b6d268b1e01","Type":"ContainerStarted","Data":"de54b21f7124b7b2443a7fb679a1674f841ddb9485cc94076b8426e0aa07cb81"}
Sep 17 21:57:54.692430 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.692336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:54.692430 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.692405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:54.692661 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.692551    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:54.692661 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.692615    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs podName:2e8cccc3-163f-4c1e-af6e-6390d5a2a694 nodeName:}" failed. No retries permitted until 2026-09-17 21:57:56.692595668 +0000 UTC m=+7.506971038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs") pod "network-metrics-daemon-xwm5p" (UID: "2e8cccc3-163f-4c1e-af6e-6390d5a2a694") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:54.693113 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.693009    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:54.693113 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.693060    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret podName:9811d2bc-c727-42e3-a514-02ca5a24be2f nodeName:}" failed. No retries permitted until 2026-09-17 21:57:56.693045172 +0000 UTC m=+7.507420528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret") pod "global-pull-secret-syncer-djhj9" (UID: "9811d2bc-c727-42e3-a514-02ca5a24be2f") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:54.713264 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.709846    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 21:57:54.793100 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:54.793061    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:54.793313 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.793233    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:57:54.793313 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.793267    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:57:54.793313 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.793281    2561 projected.go:196] Error preparing data for projected volume kube-api-access-k6mm4 for pod openshift-network-diagnostics/network-check-target-k7cfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:54.793495 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:54.793351    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4 podName:b3a32913-e0bd-40ff-b8c9-e80a8a7adc58 nodeName:}" failed. No retries permitted until 2026-09-17 21:57:56.793331092 +0000 UTC m=+7.607706470 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6mm4" (UniqueName: "kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4") pod "network-check-target-k7cfk" (UID: "b3a32913-e0bd-40ff-b8c9-e80a8a7adc58") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:55.101023 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.100895    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 21:52:53 +0000 UTC" deadline="2028-03-21 03:46:55.331764664 +0000 UTC"
Sep 17 21:57:55.101023 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.100954    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13205h49m0.230815359s"
Sep 17 21:57:55.130742 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.129827    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:55.130742 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:55.129948    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:57:55.130742 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.130374    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:55.130742 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:55.130524    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:57:55.130742 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.130588    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:55.130742 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:55.130702    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:57:55.174335 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.174300    2561 generic.go:356] "Generic (PLEG): container finished" podID="7866f8af40529517a43d922924cc06dc" containerID="2473d1f22b5194ad703c0fd2f100e98451d971a1f59dedfb66d4deb06d20312d" exitCode=0
Sep 17 21:57:55.174495 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.174387    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal" event={"ID":"7866f8af40529517a43d922924cc06dc","Type":"ContainerDied","Data":"2473d1f22b5194ad703c0fd2f100e98451d971a1f59dedfb66d4deb06d20312d"}
Sep 17 21:57:55.206800 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.206336    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal" event={"ID":"cb7e1d05bdbb0cfb0bbd514a8ce678b1","Type":"ContainerStarted","Data":"256cadb08dbca1bf5ccbe28f59e97f4ed0c58c54f8aa1ca5a8eab32f2da3d169"}
Sep 17 21:57:55.219293 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:55.219228    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-69.ec2.internal" podStartSLOduration=3.219208193 podStartE2EDuration="3.219208193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:57:55.218334943 +0000 UTC m=+6.032710321" watchObservedRunningTime="2026-09-17 21:57:55.219208193 +0000 UTC m=+6.033583577"
Sep 17 21:57:56.249365 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:56.249300    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal" event={"ID":"7866f8af40529517a43d922924cc06dc","Type":"ContainerStarted","Data":"bffe231bb35a23e364bd5ec4dacdc487da192cb7eb9f9d6b136021098c87f3c0"}
Sep 17 21:57:56.710750 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:56.710660    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:56.710908 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:56.710753    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:56.710908 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.710872    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:56.711020 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.710933    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret podName:9811d2bc-c727-42e3-a514-02ca5a24be2f nodeName:}" failed. No retries permitted until 2026-09-17 21:58:00.710915373 +0000 UTC m=+11.525290747 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret") pod "global-pull-secret-syncer-djhj9" (UID: "9811d2bc-c727-42e3-a514-02ca5a24be2f") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:57:56.711367 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.711348    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:56.711477 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.711405    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs podName:2e8cccc3-163f-4c1e-af6e-6390d5a2a694 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:00.711388523 +0000 UTC m=+11.525763901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs") pod "network-metrics-daemon-xwm5p" (UID: "2e8cccc3-163f-4c1e-af6e-6390d5a2a694") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:57:56.811794 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:56.811699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:56.811954 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.811883    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:57:56.811954 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.811911    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:57:56.811954 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.811926    2561 projected.go:196] Error preparing data for projected volume kube-api-access-k6mm4 for pod openshift-network-diagnostics/network-check-target-k7cfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:56.812114 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:56.811987    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4 podName:b3a32913-e0bd-40ff-b8c9-e80a8a7adc58 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:00.811969091 +0000 UTC m=+11.626344449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6mm4" (UniqueName: "kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4") pod "network-check-target-k7cfk" (UID: "b3a32913-e0bd-40ff-b8c9-e80a8a7adc58") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:57:57.127987 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:57.127913    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:57.128129 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:57.128047    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:57:57.128129 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:57.128066    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:57.128242 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:57.128072    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:57.128242 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:57.128174    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:57:57.128358 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:57.128235    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:57:59.127389 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:59.127358    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:57:59.127805 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:59.127358    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:57:59.127805 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:59.127501    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:57:59.127805 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:57:59.127514    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:57:59.127805 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:59.127612    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:57:59.127805 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:57:59.127687    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:00.740489 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:00.740433    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:00.740974 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:00.740510    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:00.740974 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.740666    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:58:00.740974 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.740735    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs podName:2e8cccc3-163f-4c1e-af6e-6390d5a2a694 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:08.740716722 +0000 UTC m=+19.555092080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs") pod "network-metrics-daemon-xwm5p" (UID: "2e8cccc3-163f-4c1e-af6e-6390d5a2a694") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:58:00.740974 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.740763    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:58:00.740974 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.740831    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret podName:9811d2bc-c727-42e3-a514-02ca5a24be2f nodeName:}" failed. No retries permitted until 2026-09-17 21:58:08.740813444 +0000 UTC m=+19.555188815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret") pod "global-pull-secret-syncer-djhj9" (UID: "9811d2bc-c727-42e3-a514-02ca5a24be2f") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:58:00.841527 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:00.841489    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:00.841735 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.841717    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:58:00.841811 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.841744    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:58:00.841811 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.841757    2561 projected.go:196] Error preparing data for projected volume kube-api-access-k6mm4 for pod openshift-network-diagnostics/network-check-target-k7cfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:58:00.841913 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:00.841820    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4 podName:b3a32913-e0bd-40ff-b8c9-e80a8a7adc58 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:08.841801116 +0000 UTC m=+19.656176485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6mm4" (UniqueName: "kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4") pod "network-check-target-k7cfk" (UID: "b3a32913-e0bd-40ff-b8c9-e80a8a7adc58") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:58:01.131881 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:01.131240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:01.131881 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:01.131366    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:01.131881 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:01.131419    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:01.131881 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:01.131464    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:01.131881 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:01.131496    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:01.131881 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:01.131554    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:03.127186 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:03.127156    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:03.127644 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:03.127160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:03.127644 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:03.127268    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:03.127644 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:03.127343    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:03.127644 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:03.127167    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:03.127644 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:03.127498    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:05.127371 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:05.127332    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:05.127371 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:05.127352    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:05.127875 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:05.127465    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:05.127875 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:05.127551    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:05.127875 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:05.127551    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:05.127875 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:05.127635    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:07.127879 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:07.127848    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:07.128386 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:07.127848    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:07.128386 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:07.127965    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:07.128386 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:07.127851    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:07.128386 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:07.128068    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:07.128386 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:07.128083    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:08.798153 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:08.798117    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:08.798591 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:08.798167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:08.798591 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.798269    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:58:08.798591 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.798339    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret podName:9811d2bc-c727-42e3-a514-02ca5a24be2f nodeName:}" failed. No retries permitted until 2026-09-17 21:58:24.798318766 +0000 UTC m=+35.612694136 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret") pod "global-pull-secret-syncer-djhj9" (UID: "9811d2bc-c727-42e3-a514-02ca5a24be2f") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:58:08.798591 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.798270    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:58:08.798591 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.798430    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs podName:2e8cccc3-163f-4c1e-af6e-6390d5a2a694 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:24.798408283 +0000 UTC m=+35.612783658 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs") pod "network-metrics-daemon-xwm5p" (UID: "2e8cccc3-163f-4c1e-af6e-6390d5a2a694") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:58:08.898664 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:08.898631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:08.898841 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.898820    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:58:08.898914 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.898849    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:58:08.898914 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.898864    2561 projected.go:196] Error preparing data for projected volume kube-api-access-k6mm4 for pod openshift-network-diagnostics/network-check-target-k7cfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:58:08.898987 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:08.898919    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4 podName:b3a32913-e0bd-40ff-b8c9-e80a8a7adc58 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:24.898905103 +0000 UTC m=+35.713280474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6mm4" (UniqueName: "kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4") pod "network-check-target-k7cfk" (UID: "b3a32913-e0bd-40ff-b8c9-e80a8a7adc58") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:58:09.127157 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:09.127068    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:09.127366 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:09.127068    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:09.127366 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:09.127195    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:09.127366 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:09.127340    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:09.127531 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:09.127375    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:09.127531 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:09.127443    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:11.127402 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:11.127375    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:11.127788 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:11.127441    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:11.127788 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:11.127455    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:11.127788 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:11.127534    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:11.127788 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:11.127583    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:11.127788 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:11.127672    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:12.282659 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.282304    2561 generic.go:356] "Generic (PLEG): container finished" podID="f10417bd-9a32-43ea-be80-6a2b4a414203" containerID="25928b434bbf4089829dda72fed89ab668bc5a293a103e6e480b06005d3037c1" exitCode=0
Sep 17 21:58:12.283454 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.282388    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerDied","Data":"25928b434bbf4089829dda72fed89ab668bc5a293a103e6e480b06005d3037c1"}
Sep 17 21:58:12.284736 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.284707    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hx8mn" event={"ID":"5fe371e6-2152-4ec7-8751-5b6d268b1e01","Type":"ContainerStarted","Data":"640af1b2bb3f733868118ac3c599b576e475b8351d78208b1e806850ab0b650d"}
Sep 17 21:58:12.286524 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.286501    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-msktq" event={"ID":"87535f94-4b13-4fb8-a307-eb2fdf5ec1d0","Type":"ContainerStarted","Data":"198240e978059a9237e402384e1dc0dae61aa5230cf2a8b087bd47178d44b3f2"}
Sep 17 21:58:12.288421 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.288369    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6" event={"ID":"20010908-1f7c-4c73-98aa-8598bfced30d","Type":"ContainerStarted","Data":"c1a7278281f9f885f8bdbd649009c6149e2793c0feed724a3ab3bb084eb62c66"}
Sep 17 21:58:12.290108 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.290079    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tpvqh" event={"ID":"dbd374c7-122a-4732-808f-9777edcf8fa9","Type":"ContainerStarted","Data":"ea5e3f483bd3de227d2da7ceb4b4e7d5f95dc4402f8190cd603f214c158510a6"}
Sep 17 21:58:12.291799 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.291773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6swmf" event={"ID":"e536b96a-1b2e-4b48-96d6-128570aca0cc","Type":"ContainerStarted","Data":"be6bed6a3c095bad474d656cd1746d49964d7f4e357996c362710b1e878ea706"}
Sep 17 21:58:12.295653 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.295633    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 21:58:12.296111 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.296079    2561 generic.go:356] "Generic (PLEG): container finished" podID="67602f97-3cbe-4453-aac5-114689901c4a" containerID="568a52c2fd065118658bc642bfe8eec781bfeda3abaef1314977b5d161532075" exitCode=1
Sep 17 21:58:12.296196 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.296152    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"d8d53137c5ebb7f3bdc413ccbcd98207f36d5ee077bb6bd8128435fb44a86a6d"}
Sep 17 21:58:12.296278 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.296222    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"3e3498c1170b129ca667e4a97d358cc9984e97c716bda5b3e18da419d35aca20"}
Sep 17 21:58:12.296338 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.296296    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"2111687d2cae21ca478745209c4a9e0afc10bb5585aa7a3c3862da223181ab4a"}
Sep 17 21:58:12.296385 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.296347    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"57d295ea1da410cd1e89b31dba7ef83febd541ddeb82a245da0a6cba95995868"}
Sep 17 21:58:12.296450 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.296394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerDied","Data":"568a52c2fd065118658bc642bfe8eec781bfeda3abaef1314977b5d161532075"}
Sep 17 21:58:12.296503 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.296447    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"eb23ce434dc1dafe87872e326e7b44912e32ec16c4b39db5772fbd05ea41e128"}
Sep 17 21:58:12.298781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.298758    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bxzz8" event={"ID":"776e2517-6042-4c45-b5ce-267e540962ad","Type":"ContainerStarted","Data":"f73f7e8f2c7d6fdc0e9c27ab1cd7fbcbddb06fc64094aeb18257360c496d015c"}
Sep 17 21:58:12.304626 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.304577    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-69.ec2.internal" podStartSLOduration=20.304561659 podStartE2EDuration="20.304561659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:57:56.264563158 +0000 UTC m=+7.078938536" watchObservedRunningTime="2026-09-17 21:58:12.304561659 +0000 UTC m=+23.118937040"
Sep 17 21:58:12.316305 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.316270    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tpvqh" podStartSLOduration=11.524150481 podStartE2EDuration="21.316241046s" totalImagesPullingTime="9.792090565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:12.315964518 +0000 UTC m=+23.130339894" watchObservedRunningTime="2026-09-17 21:58:12.316241046 +0000 UTC m=+23.130616422"
Sep 17 21:58:12.328626 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.328592    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6swmf" podStartSLOduration=4.060419091 podStartE2EDuration="21.328581773s" totalImagesPullingTime="17.268162682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:12.328290109 +0000 UTC m=+23.142665482" watchObservedRunningTime="2026-09-17 21:58:12.328581773 +0000 UTC m=+23.142957150"
Sep 17 21:58:12.341078 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.341045    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-msktq" podStartSLOduration=4.080064673 podStartE2EDuration="21.341034493s" totalImagesPullingTime="17.26096982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:12.340550435 +0000 UTC m=+23.154925813" watchObservedRunningTime="2026-09-17 21:58:12.341034493 +0000 UTC m=+23.155409887"
Sep 17 21:58:12.354980 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.354945    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bxzz8" podStartSLOduration=4.050328599 podStartE2EDuration="21.354935779s" totalImagesPullingTime="17.30460718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:12.354696029 +0000 UTC m=+23.169071408" watchObservedRunningTime="2026-09-17 21:58:12.354935779 +0000 UTC m=+23.169311155"
Sep 17 21:58:12.370000 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.369970    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hx8mn" podStartSLOduration=4.092341759 podStartE2EDuration="21.369957409s" totalImagesPullingTime="17.27761565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:12.369643506 +0000 UTC m=+23.184018894" watchObservedRunningTime="2026-09-17 21:58:12.369957409 +0000 UTC m=+23.184332785"
Sep 17 21:58:12.554541 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:12.554517    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 21:58:13.127965 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.127892    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:13.128171 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.127892    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:13.128171 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.128014    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:13.128171 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:13.128005    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:13.128171 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:13.128132    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:13.128404 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:13.128206    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:13.135084 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.134990    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T21:58:12.554533791Z","UUID":"bcc034a1-3606-418f-8e03-fb6017fbaaf5","Handler":null,"Name":"","Endpoint":""}
Sep 17 21:58:13.136946 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.136926    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 21:58:13.137071 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.136956    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 21:58:13.303736 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.303697    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6" event={"ID":"20010908-1f7c-4c73-98aa-8598bfced30d","Type":"ContainerStarted","Data":"e4c72ba97b59fea0e4447cc490028bc046542199a2392b34e3a8ad0c9bf2fbed"}
Sep 17 21:58:13.305386 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:13.305353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cbzc2" event={"ID":"aa8136e8-83ac-4e34-ae35-c6c11d62f18f","Type":"ContainerStarted","Data":"f1760007252930f13cd9b87f80d8b8a244e1af055befd841bceaafe9062e9e85"}
Sep 17 21:58:14.167211 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:14.166975    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:58:14.168042 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:14.168015    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:58:14.182024 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:14.181977    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cbzc2" podStartSLOduration=5.938211617 podStartE2EDuration="23.181958839s" totalImagesPullingTime="17.243747222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:13.319886791 +0000 UTC m=+24.134262168" watchObservedRunningTime="2026-09-17 21:58:14.181958839 +0000 UTC m=+24.996334216"
Sep 17 21:58:14.309896 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:14.309865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6" event={"ID":"20010908-1f7c-4c73-98aa-8598bfced30d","Type":"ContainerStarted","Data":"ae24f2dc3521ddcf78fa58062a1017ff9ed3cf61e62567d63f531112af31afc0"}
Sep 17 21:58:14.336749 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:14.336697    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r84c6" podStartSLOduration=4.011236003 podStartE2EDuration="23.336678196s" totalImagesPullingTime="19.325442193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:14.335959306 +0000 UTC m=+25.150334696" watchObservedRunningTime="2026-09-17 21:58:14.336678196 +0000 UTC m=+25.151053579"
Sep 17 21:58:15.127939 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:15.127816    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:15.127939 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:15.127817    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:15.128148 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:15.127951    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:15.128148 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:15.127817    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:15.128148 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:15.128036    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:15.128148 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:15.128106    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:15.316013 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:15.315987    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 21:58:15.316471 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:15.316385    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"306fd76a61702b9aae895de4a3c4aab2d55d19d2d9382632cb21a32302e939f5"}
Sep 17 21:58:15.316695 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:15.316679    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 21:58:17.127502 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.127331    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:17.128159 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.127338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:17.128159 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:17.127564    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:17.128159 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.127366    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:17.128159 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:17.127644    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:17.128159 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:17.127736    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:17.323177 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.323152    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 21:58:17.323594 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.323542    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"6560002aca6c24d5b94894dd3e680c658810541e6d2fab673e4832c1e4826cde"}
Sep 17 21:58:17.323778 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.323762    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:58:17.323988 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.323960    2561 scope.go:117] "RemoveContainer" containerID="568a52c2fd065118658bc642bfe8eec781bfeda3abaef1314977b5d161532075"
Sep 17 21:58:17.325364 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.325345    2561 generic.go:356] "Generic (PLEG): container finished" podID="f10417bd-9a32-43ea-be80-6a2b4a414203" containerID="e2bdd1fcc2cebb7f6d3e66c9080d82ff40fcca9a22a10f7ab3bfcdd7221c2fa2" exitCode=0
Sep 17 21:58:17.325451 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.325377    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerDied","Data":"e2bdd1fcc2cebb7f6d3e66c9080d82ff40fcca9a22a10f7ab3bfcdd7221c2fa2"}
Sep 17 21:58:17.340468 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:17.340451    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:58:18.331130 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.331104    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 21:58:18.331539 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.331448    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" event={"ID":"67602f97-3cbe-4453-aac5-114689901c4a","Type":"ContainerStarted","Data":"3c71277d3fb40a90658a22b6fa52d3af5fafbc6615c17e5c1340bad24fc04f05"}
Sep 17 21:58:18.331581 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.331547    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 21:58:18.331830 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.331807    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:58:18.348523 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.348496    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:58:18.362481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.360506    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k7cfk"]
Sep 17 21:58:18.362481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.360601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xwm5p"]
Sep 17 21:58:18.362481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.360711    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:18.362481 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:18.360848    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:18.362481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.361169    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:18.362481 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:18.361313    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:18.362481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.362405    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djhj9"]
Sep 17 21:58:18.363511 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.363394    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:18.363702 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:18.363674    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:18.363790 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:18.363719    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" podStartSLOduration=10.070346403 podStartE2EDuration="27.36370419s" totalImagesPullingTime="17.293357787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:18.363472923 +0000 UTC m=+29.177848322" watchObservedRunningTime="2026-09-17 21:58:18.36370419 +0000 UTC m=+29.178079567"
Sep 17 21:58:19.335992 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:19.335812    2561 generic.go:356] "Generic (PLEG): container finished" podID="f10417bd-9a32-43ea-be80-6a2b4a414203" containerID="6ac39cf8015b5ca54d155119f64133efc7b8e6378747cc553de37297b629a50d" exitCode=0
Sep 17 21:58:19.336476 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:19.335896    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerDied","Data":"6ac39cf8015b5ca54d155119f64133efc7b8e6378747cc553de37297b629a50d"}
Sep 17 21:58:19.336476 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:19.336170    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 21:58:19.996562 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:19.996524    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:58:19.996723 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:19.996658    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 21:58:19.997291 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:19.997271    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tpvqh"
Sep 17 21:58:20.126758 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:20.126728    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:20.126913 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:20.126821    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:20.126913 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:20.126816    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:20.126913 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:20.126901    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:20.126913 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:20.126827    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:20.127104 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:20.127018    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:20.337873 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:20.337802    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 21:58:20.427915 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:20.427897    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:58:21.342076 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:21.342039    2561 generic.go:356] "Generic (PLEG): container finished" podID="f10417bd-9a32-43ea-be80-6a2b4a414203" containerID="905b9c112a0333464fc5390c966e28db12f9540a8418af5e85fb6e3fa58031b2" exitCode=0
Sep 17 21:58:21.342600 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:21.342127    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerDied","Data":"905b9c112a0333464fc5390c966e28db12f9540a8418af5e85fb6e3fa58031b2"}
Sep 17 21:58:21.355924 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:21.355891    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t" podUID="67602f97-3cbe-4453-aac5-114689901c4a" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 17 21:58:22.127707 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:22.127675    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:22.127707 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:22.127702    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:22.127911 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:22.127797    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:22.127911 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:22.127827    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:22.127911 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:22.127889    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:22.128095 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:22.127950    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:24.127054 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:24.127022    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:24.127054 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:24.127043    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:24.127657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:24.127029    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:24.127657 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.127131    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7cfk" podUID="b3a32913-e0bd-40ff-b8c9-e80a8a7adc58"
Sep 17 21:58:24.127657 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.127222    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djhj9" podUID="9811d2bc-c727-42e3-a514-02ca5a24be2f"
Sep 17 21:58:24.127657 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.127347    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xwm5p" podUID="2e8cccc3-163f-4c1e-af6e-6390d5a2a694"
Sep 17 21:58:24.831241 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:24.831153    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:24.831241 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:24.831218    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:24.831502 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.831300    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 21:58:24.831502 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.831341    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:58:24.831502 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.831373    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret podName:9811d2bc-c727-42e3-a514-02ca5a24be2f nodeName:}" failed. No retries permitted until 2026-09-17 21:58:56.83135147 +0000 UTC m=+67.645726840 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret") pod "global-pull-secret-syncer-djhj9" (UID: "9811d2bc-c727-42e3-a514-02ca5a24be2f") : object "kube-system"/"original-pull-secret" not registered
Sep 17 21:58:24.831502 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.831394    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs podName:2e8cccc3-163f-4c1e-af6e-6390d5a2a694 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:56.831383323 +0000 UTC m=+67.645758678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs") pod "network-metrics-daemon-xwm5p" (UID: "2e8cccc3-163f-4c1e-af6e-6390d5a2a694") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 21:58:24.931951 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:24.931914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:24.932124 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.932090    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 21:58:24.932124 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.932110    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 21:58:24.932124 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.932121    2561 projected.go:196] Error preparing data for projected volume kube-api-access-k6mm4 for pod openshift-network-diagnostics/network-check-target-k7cfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:58:24.932288 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:24.932188    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4 podName:b3a32913-e0bd-40ff-b8c9-e80a8a7adc58 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:56.932170643 +0000 UTC m=+67.746546002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6mm4" (UniqueName: "kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4") pod "network-check-target-k7cfk" (UID: "b3a32913-e0bd-40ff-b8c9-e80a8a7adc58") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 21:58:25.339436 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.339409    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-69.ec2.internal" event="NodeReady"
Sep 17 21:58:25.339871 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.339521    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 21:58:25.370023 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.369937    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85d57b5bc-rnmnl"]
Sep 17 21:58:25.384702 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.384679    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gbdmh"]
Sep 17 21:58:25.384847 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.384826    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.387159 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.387141    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 21:58:25.387272 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.387172    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nrw4l\""
Sep 17 21:58:25.387482 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.387462    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 21:58:25.387482 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.387479    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 21:58:25.395883 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.395854    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 21:58:25.399018 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.398999    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bw55n"]
Sep 17 21:58:25.399135 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.399120    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.401481 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.401461    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 21:58:25.401553 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.401508    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 21:58:25.401610 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.401562    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7dc29\""
Sep 17 21:58:25.417219 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.417200    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85d57b5bc-rnmnl"]
Sep 17 21:58:25.417326 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.417229    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bw55n"]
Sep 17 21:58:25.417326 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.417264    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gbdmh"]
Sep 17 21:58:25.417394 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.417332    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:25.419601 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.419557    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 21:58:25.419601 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.419586    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 21:58:25.419601 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.419595    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4blcd\""
Sep 17 21:58:25.419759 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.419562    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 21:58:25.435436 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435407    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ddcb190d-cc99-43e6-b1ba-59a1b0450888-config-volume\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.435531 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435442    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgq2c\" (UniqueName: \"kubernetes.io/projected/ddcb190d-cc99-43e6-b1ba-59a1b0450888-kube-api-access-sgq2c\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.435531 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435473    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-bound-sa-token\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.435628 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435526    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-image-registry-private-configuration\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.435628 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435547    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.435628 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435600    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-trusted-ca\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.435832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435648    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkzhh\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-kube-api-access-vkzhh\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.435832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435666    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-certificates\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.435832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435683    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxcls\" (UniqueName: \"kubernetes.io/projected/e6a474d1-7b6b-4408-945d-0125b4f216ec-kube-api-access-pxcls\") pod \"ingress-canary-bw55n\" (UID: \"e6a474d1-7b6b-4408-945d-0125b4f216ec\") " pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:25.435832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435702    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.435832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435744    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8a6403-fbe9-483e-b22b-181fc07c76ad-ca-trust-extracted\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.435832 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435825    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6a474d1-7b6b-4408-945d-0125b4f216ec-cert\") pod \"ingress-canary-bw55n\" (UID: \"e6a474d1-7b6b-4408-945d-0125b4f216ec\") " pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:25.436063 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435852    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-installation-pull-secrets\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.436063 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.435876    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcb190d-cc99-43e6-b1ba-59a1b0450888-tmp-dir\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.536155 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536122    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-image-registry-private-configuration\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.536155 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536157    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.536401 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536184    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-trusted-ca\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.536401 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzhh\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-kube-api-access-vkzhh\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.536401 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536221    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-certificates\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.536401 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:25.536309    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 21:58:25.536401 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:25.536329    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d57b5bc-rnmnl: secret "image-registry-tls" not found
Sep 17 21:58:25.536401 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxcls\" (UniqueName: \"kubernetes.io/projected/e6a474d1-7b6b-4408-945d-0125b4f216ec-kube-api-access-pxcls\") pod \"ingress-canary-bw55n\" (UID: \"e6a474d1-7b6b-4408-945d-0125b4f216ec\") " pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:25.536401 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:25.536421    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls podName:9b8a6403-fbe9-483e-b22b-181fc07c76ad nodeName:}" failed. No retries permitted until 2026-09-17 21:58:26.036400648 +0000 UTC m=+36.850776006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls") pod "image-registry-85d57b5bc-rnmnl" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad") : secret "image-registry-tls" not found
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536464    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8a6403-fbe9-483e-b22b-181fc07c76ad-ca-trust-extracted\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:25.536480    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6a474d1-7b6b-4408-945d-0125b4f216ec-cert\") pod \"ingress-canary-bw55n\" (UID: \"e6a474d1-7b6b-4408-945d-0125b4f216ec\") " pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:25.536527    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls podName:ddcb190d-cc99-43e6-b1ba-59a1b0450888 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:26.03651077 +0000 UTC m=+36.850886131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls") pod "dns-default-gbdmh" (UID: "ddcb190d-cc99-43e6-b1ba-59a1b0450888") : secret "dns-default-metrics-tls" not found
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536559    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-installation-pull-secrets\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536587    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcb190d-cc99-43e6-b1ba-59a1b0450888-tmp-dir\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536637    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ddcb190d-cc99-43e6-b1ba-59a1b0450888-config-volume\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.536705 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgq2c\" (UniqueName: \"kubernetes.io/projected/ddcb190d-cc99-43e6-b1ba-59a1b0450888-kube-api-access-sgq2c\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.537128 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536709    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-bound-sa-token\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.537128 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536829    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-certificates\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.537128 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536889    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8a6403-fbe9-483e-b22b-181fc07c76ad-ca-trust-extracted\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.537128 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.536974    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcb190d-cc99-43e6-b1ba-59a1b0450888-tmp-dir\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.537473 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.537332    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-trusted-ca\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.537473 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.537350    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ddcb190d-cc99-43e6-b1ba-59a1b0450888-config-volume\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.540233 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.540210    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6a474d1-7b6b-4408-945d-0125b4f216ec-cert\") pod \"ingress-canary-bw55n\" (UID: \"e6a474d1-7b6b-4408-945d-0125b4f216ec\") " pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:25.540346 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.540283    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-image-registry-private-configuration\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.540346 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.540286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-installation-pull-secrets\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.546463 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.546436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-bound-sa-token\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.546554 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.546530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxcls\" (UniqueName: \"kubernetes.io/projected/e6a474d1-7b6b-4408-945d-0125b4f216ec-kube-api-access-pxcls\") pod \"ingress-canary-bw55n\" (UID: \"e6a474d1-7b6b-4408-945d-0125b4f216ec\") " pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:25.546660 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.546644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzhh\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-kube-api-access-vkzhh\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:25.546964 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.546948    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgq2c\" (UniqueName: \"kubernetes.io/projected/ddcb190d-cc99-43e6-b1ba-59a1b0450888-kube-api-access-sgq2c\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:25.725119 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:25.725096    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bw55n"
Sep 17 21:58:26.039178 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.039078    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bw55n"]
Sep 17 21:58:26.041532 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.041509    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:26.041643 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.041625    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:26.041689 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:26.041653    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 21:58:26.041729 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:26.041716    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls podName:ddcb190d-cc99-43e6-b1ba-59a1b0450888 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:27.041698093 +0000 UTC m=+37.856073465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls") pod "dns-default-gbdmh" (UID: "ddcb190d-cc99-43e6-b1ba-59a1b0450888") : secret "dns-default-metrics-tls" not found
Sep 17 21:58:26.041774 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:26.041748    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 21:58:26.041774 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:26.041761    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d57b5bc-rnmnl: secret "image-registry-tls" not found
Sep 17 21:58:26.041840 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:26.041824    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls podName:9b8a6403-fbe9-483e-b22b-181fc07c76ad nodeName:}" failed. No retries permitted until 2026-09-17 21:58:27.04181007 +0000 UTC m=+37.856185443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls") pod "image-registry-85d57b5bc-rnmnl" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad") : secret "image-registry-tls" not found
Sep 17 21:58:26.043135 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:26.043116    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a474d1_7b6b_4408_945d_0125b4f216ec.slice/crio-6fefbcd7b4c98b3d76fc8d7ef32125e1f4a8a4c4443b077945c28f85fc72625c WatchSource:0}: Error finding container 6fefbcd7b4c98b3d76fc8d7ef32125e1f4a8a4c4443b077945c28f85fc72625c: Status 404 returned error can't find the container with id 6fefbcd7b4c98b3d76fc8d7ef32125e1f4a8a4c4443b077945c28f85fc72625c
Sep 17 21:58:26.126784 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.126758    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:26.126921 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.126869    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:26.126921 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.126898    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:26.129477 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.129454    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 21:58:26.129600 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.129557    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 21:58:26.129600 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.129594    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 21:58:26.129697 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.129593    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 21:58:26.130493 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.130476    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-278n4\""
Sep 17 21:58:26.130582 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.130478    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nbkcq\""
Sep 17 21:58:26.356915 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.356830    2561 generic.go:356] "Generic (PLEG): container finished" podID="f10417bd-9a32-43ea-be80-6a2b4a414203" containerID="fae4105474325b8853444dbb0a30d53c8f85e1311d6952dc86ff2b021d1abfdf" exitCode=0
Sep 17 21:58:26.356915 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.356901    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerDied","Data":"fae4105474325b8853444dbb0a30d53c8f85e1311d6952dc86ff2b021d1abfdf"}
Sep 17 21:58:26.358135 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:26.358115    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bw55n" event={"ID":"e6a474d1-7b6b-4408-945d-0125b4f216ec","Type":"ContainerStarted","Data":"6fefbcd7b4c98b3d76fc8d7ef32125e1f4a8a4c4443b077945c28f85fc72625c"}
Sep 17 21:58:27.048696 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:27.048664    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:27.048937 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:27.048729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:27.048937 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:27.048829    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 21:58:27.048937 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:27.048849    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d57b5bc-rnmnl: secret "image-registry-tls" not found
Sep 17 21:58:27.048937 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:27.048849    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 21:58:27.048937 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:27.048911    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls podName:9b8a6403-fbe9-483e-b22b-181fc07c76ad nodeName:}" failed. No retries permitted until 2026-09-17 21:58:29.048890249 +0000 UTC m=+39.863265630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls") pod "image-registry-85d57b5bc-rnmnl" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad") : secret "image-registry-tls" not found
Sep 17 21:58:27.048937 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:27.048930    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls podName:ddcb190d-cc99-43e6-b1ba-59a1b0450888 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:29.04892131 +0000 UTC m=+39.863296670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls") pod "dns-default-gbdmh" (UID: "ddcb190d-cc99-43e6-b1ba-59a1b0450888") : secret "dns-default-metrics-tls" not found
Sep 17 21:58:27.365128 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:27.364893    2561 generic.go:356] "Generic (PLEG): container finished" podID="f10417bd-9a32-43ea-be80-6a2b4a414203" containerID="5885848e4ee53502df85005b7256bf6b0ceac777eb3ea3eacf8e68793df2f5d8" exitCode=0
Sep 17 21:58:27.365128 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:27.364977    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerDied","Data":"5885848e4ee53502df85005b7256bf6b0ceac777eb3ea3eacf8e68793df2f5d8"}
Sep 17 21:58:28.370778 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:28.370706    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" event={"ID":"f10417bd-9a32-43ea-be80-6a2b4a414203","Type":"ContainerStarted","Data":"d84a8cdb8d535116f3eb14ef796926dbba61263656d0ad0ebdfc956d25a928d5"}
Sep 17 21:58:28.391449 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:28.391407    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5qc2s" podStartSLOduration=9.517394901 podStartE2EDuration="37.391392431s" totalImagesPullingTime="27.87399753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:28.390474917 +0000 UTC m=+39.204850294" watchObservedRunningTime="2026-09-17 21:58:28.391392431 +0000 UTC m=+39.205767786"
Sep 17 21:58:29.064437 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:29.064346    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:29.064437 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:29.064413    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:29.064641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:29.064538    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 21:58:29.064641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:29.064569    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d57b5bc-rnmnl: secret "image-registry-tls" not found
Sep 17 21:58:29.064641 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:29.064549    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 21:58:29.064752 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:29.064627    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls podName:9b8a6403-fbe9-483e-b22b-181fc07c76ad nodeName:}" failed. No retries permitted until 2026-09-17 21:58:33.064611898 +0000 UTC m=+43.878987259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls") pod "image-registry-85d57b5bc-rnmnl" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad") : secret "image-registry-tls" not found
Sep 17 21:58:29.064752 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:29.064693    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls podName:ddcb190d-cc99-43e6-b1ba-59a1b0450888 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:33.064677566 +0000 UTC m=+43.879052925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls") pod "dns-default-gbdmh" (UID: "ddcb190d-cc99-43e6-b1ba-59a1b0450888") : secret "dns-default-metrics-tls" not found
Sep 17 21:58:29.374975 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:29.374899    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bw55n" event={"ID":"e6a474d1-7b6b-4408-945d-0125b4f216ec","Type":"ContainerStarted","Data":"f19d3b3d7029c6a7e1df2c6155ba9890b2ba72107844a425cfc89791446aaf32"}
Sep 17 21:58:29.389865 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:29.389820    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bw55n" podStartSLOduration=1.74938992 podStartE2EDuration="4.389807076s" totalImagesPullingTime="2.640417156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:29.388515746 +0000 UTC m=+40.202891124" watchObservedRunningTime="2026-09-17 21:58:29.389807076 +0000 UTC m=+40.204182484"
Sep 17 21:58:33.089231 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.089197    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:33.089612 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.089261    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:33.089612 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.089342    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 21:58:33.089612 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.089355    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 21:58:33.089612 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.089362    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d57b5bc-rnmnl: secret "image-registry-tls" not found
Sep 17 21:58:33.089612 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.089408    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls podName:ddcb190d-cc99-43e6-b1ba-59a1b0450888 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:41.089394749 +0000 UTC m=+51.903770118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls") pod "dns-default-gbdmh" (UID: "ddcb190d-cc99-43e6-b1ba-59a1b0450888") : secret "dns-default-metrics-tls" not found
Sep 17 21:58:33.089612 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.089420    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls podName:9b8a6403-fbe9-483e-b22b-181fc07c76ad nodeName:}" failed. No retries permitted until 2026-09-17 21:58:41.089414386 +0000 UTC m=+51.903789740 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls") pod "image-registry-85d57b5bc-rnmnl" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad") : secret "image-registry-tls" not found
Sep 17 21:58:33.334993 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.334934    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xnfbw"]
Sep 17 21:58:33.338019 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.338001    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.340019 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.339970    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 21:58:33.340436 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.340422    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 21:58:33.340525 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.340441    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 21:58:33.340984 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.340962    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tsmwz\""
Sep 17 21:58:33.340984 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.340976    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 21:58:33.348407 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.348388    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xnfbw"]
Sep 17 21:58:33.391054 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.391030    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.391160 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.391057    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1a1026a-0e11-4e8b-8e43-1f433fe41480-crio-socket\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.391160 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.391076    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a1026a-0e11-4e8b-8e43-1f433fe41480-data-volume\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.391160 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.391100    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ngwr\" (UniqueName: \"kubernetes.io/projected/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-api-access-8ngwr\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.391274 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.391166    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.492399 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.492373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.492498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.492441    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.492498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.492475    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1a1026a-0e11-4e8b-8e43-1f433fe41480-crio-socket\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.492498 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.492491    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a1026a-0e11-4e8b-8e43-1f433fe41480-data-volume\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.492632 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.492495    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 21:58:33.492632 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.492510    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ngwr\" (UniqueName: \"kubernetes.io/projected/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-api-access-8ngwr\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.492632 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.492581    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:33.992559695 +0000 UTC m=+44.806935070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : configmap "kube-rbac-proxy" not found
Sep 17 21:58:33.492771 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.492635    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:33.492771 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.492681    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1a1026a-0e11-4e8b-8e43-1f433fe41480-crio-socket\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.492771 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.492701    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:33.992683673 +0000 UTC m=+44.807059043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:33.492893 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.492873    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a1026a-0e11-4e8b-8e43-1f433fe41480-data-volume\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.503900 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.503877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ngwr\" (UniqueName: \"kubernetes.io/projected/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-api-access-8ngwr\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.996073 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.996027    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.996277 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:33.996114    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:33.996277 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.996181    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:33.996277 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.996202    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 21:58:33.996277 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.996267    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:34.996236378 +0000 UTC m=+45.810611733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:33.996423 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:33.996286    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:34.996279528 +0000 UTC m=+45.810654896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : configmap "kube-rbac-proxy" not found
Sep 17 21:58:34.381937 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.381870    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"]
Sep 17 21:58:34.386450 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.386427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"
Sep 17 21:58:34.388784 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.388766    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 17 21:58:34.389688 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.389669    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 17 21:58:34.389688 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.389686    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ncdx9\""
Sep 17 21:58:34.393487 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.393454    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"]
Sep 17 21:58:34.500046 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.500014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjzbd\" (UniqueName: \"kubernetes.io/projected/49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc-kube-api-access-hjzbd\") pod \"migrator-76794cbd6c-zzvgs\" (UID: \"49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"
Sep 17 21:58:34.601135 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.601102    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjzbd\" (UniqueName: \"kubernetes.io/projected/49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc-kube-api-access-hjzbd\") pod \"migrator-76794cbd6c-zzvgs\" (UID: \"49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"
Sep 17 21:58:34.608172 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.608148    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjzbd\" (UniqueName: \"kubernetes.io/projected/49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc-kube-api-access-hjzbd\") pod \"migrator-76794cbd6c-zzvgs\" (UID: \"49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"
Sep 17 21:58:34.695673 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.695652    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"
Sep 17 21:58:34.717840 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.717821    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6swmf_e536b96a-1b2e-4b48-96d6-128570aca0cc/dns-node-resolver/0.log"
Sep 17 21:58:34.982436 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:34.982364    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs"]
Sep 17 21:58:34.985581 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:34.985553    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c5740f_9bf0_41cb_90f8_2b9aa9ee51fc.slice/crio-619664c137d49d3f8e8d0067532e4db8c61fea4bfea68a69d009d4e9e46c6c7b WatchSource:0}: Error finding container 619664c137d49d3f8e8d0067532e4db8c61fea4bfea68a69d009d4e9e46c6c7b: Status 404 returned error can't find the container with id 619664c137d49d3f8e8d0067532e4db8c61fea4bfea68a69d009d4e9e46c6c7b
Sep 17 21:58:35.004331 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:35.004310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:35.004427 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:35.004368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:35.004476 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:35.004437    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 21:58:35.004476 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:35.004458    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:35.004541 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:35.004510    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:37.004495866 +0000 UTC m=+47.818871237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : configmap "kube-rbac-proxy" not found
Sep 17 21:58:35.004541 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:35.004531    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:37.00452169 +0000 UTC m=+47.818897049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:35.389570 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:35.389496    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs" event={"ID":"49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc","Type":"ContainerStarted","Data":"619664c137d49d3f8e8d0067532e4db8c61fea4bfea68a69d009d4e9e46c6c7b"}
Sep 17 21:58:35.916048 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:35.916015    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-msktq_87535f94-4b13-4fb8-a307-eb2fdf5ec1d0/node-ca/0.log"
Sep 17 21:58:36.516117 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.516088    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bw55n_e6a474d1-7b6b-4408-945d-0125b4f216ec/serve-healthcheck-canary/0.log"
Sep 17 21:58:36.668001 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.667966    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-n89cv"]
Sep 17 21:58:36.690547 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.690518    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-n89cv"]
Sep 17 21:58:36.690685 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.690605    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.692909 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.692877    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 17 21:58:36.693039 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.692940    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 17 21:58:36.693101 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.693035    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hjpk5\""
Sep 17 21:58:36.694016 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.693999    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 17 21:58:36.694070 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.694016    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 17 21:58:36.719750 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.719730    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-signing-key\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.719834 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.719785    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w5s9\" (UniqueName: \"kubernetes.io/projected/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-kube-api-access-7w5s9\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.719882 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.719867    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-signing-cabundle\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.820793 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.820723    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-signing-cabundle\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.820793 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.820784    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-signing-key\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.820963 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.820847    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w5s9\" (UniqueName: \"kubernetes.io/projected/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-kube-api-access-7w5s9\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.821443 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.821421    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-signing-cabundle\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.823182 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.823160    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-signing-key\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.830355 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.830332    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w5s9\" (UniqueName: \"kubernetes.io/projected/bd1d5b8b-ddb6-495a-9414-c8f1445ba06f-kube-api-access-7w5s9\") pod \"service-ca-55f6fc7b86-n89cv\" (UID: \"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f\") " pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:36.999628 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:36.999608    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv"
Sep 17 21:58:37.022855 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:37.022829    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:37.022954 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:37.022898    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:37.023011 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:37.022975    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 21:58:37.023011 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:37.022981    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:37.023111 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:37.023028    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:41.023010972 +0000 UTC m=+51.837386347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : configmap "kube-rbac-proxy" not found
Sep 17 21:58:37.023111 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:37.023040    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:41.023034725 +0000 UTC m=+51.837410079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:37.282380 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:37.282348    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-n89cv"]
Sep 17 21:58:37.285929 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:37.285905    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1d5b8b_ddb6_495a_9414_c8f1445ba06f.slice/crio-a925b94c5a3fce2220d74143787d116ea1f48b9af2a39f98278fd1b35e2be6b0 WatchSource:0}: Error finding container a925b94c5a3fce2220d74143787d116ea1f48b9af2a39f98278fd1b35e2be6b0: Status 404 returned error can't find the container with id a925b94c5a3fce2220d74143787d116ea1f48b9af2a39f98278fd1b35e2be6b0
Sep 17 21:58:37.395797 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:37.395732    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs" event={"ID":"49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc","Type":"ContainerStarted","Data":"40aeb223bee30fddc5c33eb86f8724746c879bc37b5c433d67844e40b1cc99fe"}
Sep 17 21:58:37.395909 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:37.395813    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs" event={"ID":"49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc","Type":"ContainerStarted","Data":"a293e8bdf95b5231af22cdcc5e7daca45394aa057f93426dd6824588b42ac73c"}
Sep 17 21:58:37.396870 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:37.396848    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv" event={"ID":"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f","Type":"ContainerStarted","Data":"a925b94c5a3fce2220d74143787d116ea1f48b9af2a39f98278fd1b35e2be6b0"}
Sep 17 21:58:37.412018 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:37.411972    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-zzvgs" podStartSLOduration=1.528591938 podStartE2EDuration="3.411957472s" totalImagesPullingTime="1.883365534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:37.411380129 +0000 UTC m=+48.225755508" watchObservedRunningTime="2026-09-17 21:58:37.411957472 +0000 UTC m=+48.226332850"
Sep 17 21:58:40.408776 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:40.408741    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv" event={"ID":"bd1d5b8b-ddb6-495a-9414-c8f1445ba06f","Type":"ContainerStarted","Data":"f0504cedf2ddf70b07d993b07ec22711dc7bb51681bdb62e2f3c28c61be7be79"}
Sep 17 21:58:40.422759 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:40.422715    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-n89cv" podStartSLOduration=1.987121245 podStartE2EDuration="4.422701535s" totalImagesPullingTime="2.43558029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:40.422366467 +0000 UTC m=+51.236741845" watchObservedRunningTime="2026-09-17 21:58:40.422701535 +0000 UTC m=+51.237076906"
Sep 17 21:58:41.053462 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:41.053425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:41.053655 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:41.053528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:41.053714 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:41.053657    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:41.053769 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:41.053718    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls podName:d1a1026a-0e11-4e8b-8e43-1f433fe41480 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:49.053699635 +0000 UTC m=+59.868074994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xnfbw" (UID: "d1a1026a-0e11-4e8b-8e43-1f433fe41480") : secret "insights-runtime-extractor-tls" not found
Sep 17 21:58:41.054444 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:41.054411    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1a1026a-0e11-4e8b-8e43-1f433fe41480-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:41.154968 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:41.154929    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:41.155128 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:41.155020    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:41.155235 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:41.155219    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 21:58:41.155323 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:41.155312    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls podName:ddcb190d-cc99-43e6-b1ba-59a1b0450888 nodeName:}" failed. No retries permitted until 2026-09-17 21:58:57.155284735 +0000 UTC m=+67.969660095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls") pod "dns-default-gbdmh" (UID: "ddcb190d-cc99-43e6-b1ba-59a1b0450888") : secret "dns-default-metrics-tls" not found
Sep 17 21:58:41.155808 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:41.155791    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 21:58:41.155861 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:41.155812    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d57b5bc-rnmnl: secret "image-registry-tls" not found
Sep 17 21:58:41.155916 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:41.155865    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls podName:9b8a6403-fbe9-483e-b22b-181fc07c76ad nodeName:}" failed. No retries permitted until 2026-09-17 21:58:57.155849486 +0000 UTC m=+67.970224849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls") pod "image-registry-85d57b5bc-rnmnl" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad") : secret "image-registry-tls" not found
Sep 17 21:58:49.113937 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:49.113899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:49.116166 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:49.116139    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1a1026a-0e11-4e8b-8e43-1f433fe41480-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnfbw\" (UID: \"d1a1026a-0e11-4e8b-8e43-1f433fe41480\") " pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:49.247024 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:49.246995    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xnfbw"
Sep 17 21:58:49.515486 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:49.515463    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xnfbw"]
Sep 17 21:58:49.517618 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:49.517588    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a1026a_0e11_4e8b_8e43_1f433fe41480.slice/crio-4155e0bd5a7e7afc76cd1ee6ac81d1cc6bf425831c036ce92c6a46559116f506 WatchSource:0}: Error finding container 4155e0bd5a7e7afc76cd1ee6ac81d1cc6bf425831c036ce92c6a46559116f506: Status 404 returned error can't find the container with id 4155e0bd5a7e7afc76cd1ee6ac81d1cc6bf425831c036ce92c6a46559116f506
Sep 17 21:58:50.436654 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:50.436621    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnfbw" event={"ID":"d1a1026a-0e11-4e8b-8e43-1f433fe41480","Type":"ContainerStarted","Data":"c079ff2bdc18ac7e51d07ff100e903f11ed7083a3f9d0a6bda9186e9239c5008"}
Sep 17 21:58:50.437097 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:50.436675    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnfbw" event={"ID":"d1a1026a-0e11-4e8b-8e43-1f433fe41480","Type":"ContainerStarted","Data":"25884778675de391a38911719a5f17d04a7c4a21d63dd53f71f3ea61acd7f2ee"}
Sep 17 21:58:50.437097 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:50.436692    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnfbw" event={"ID":"d1a1026a-0e11-4e8b-8e43-1f433fe41480","Type":"ContainerStarted","Data":"4155e0bd5a7e7afc76cd1ee6ac81d1cc6bf425831c036ce92c6a46559116f506"}
Sep 17 21:58:51.353169 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:51.353133    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-24w8t"
Sep 17 21:58:52.445608 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:52.445574    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnfbw" event={"ID":"d1a1026a-0e11-4e8b-8e43-1f433fe41480","Type":"ContainerStarted","Data":"819afdfd186589d2bd5564d7b854e757c175978759a5d3158737dceb0b3e43d0"}
Sep 17 21:58:52.461809 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:52.461759    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xnfbw" podStartSLOduration=17.562969496 podStartE2EDuration="19.461740238s" totalImagesPullingTime="1.898770742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:58:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:52.460802134 +0000 UTC m=+63.275177507" watchObservedRunningTime="2026-09-17 21:58:52.461740238 +0000 UTC m=+63.276115616"
Sep 17 21:58:56.873082 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.873048    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:56.873509 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.873109    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:56.875483 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.875464    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 21:58:56.875557 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.875540    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 21:58:56.885534 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.885504    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9811d2bc-c727-42e3-a514-02ca5a24be2f-original-pull-secret\") pod \"global-pull-secret-syncer-djhj9\" (UID: \"9811d2bc-c727-42e3-a514-02ca5a24be2f\") " pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:56.885643 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.885551    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8cccc3-163f-4c1e-af6e-6390d5a2a694-metrics-certs\") pod \"network-metrics-daemon-xwm5p\" (UID: \"2e8cccc3-163f-4c1e-af6e-6390d5a2a694\") " pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:56.956836 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.956810    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-wb4nh"]
Sep 17 21:58:56.959847 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.959831    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-wb4nh"
Sep 17 21:58:56.962369 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.962339    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 21:58:56.962493 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.962404    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 21:58:56.962493 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.962405    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-55jmc\""
Sep 17 21:58:56.967868 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.967845    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85d57b5bc-rnmnl"]
Sep 17 21:58:56.968084 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:58:56.968059    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl" podUID="9b8a6403-fbe9-483e-b22b-181fc07c76ad"
Sep 17 21:58:56.971276 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.971241    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-wb4nh"]
Sep 17 21:58:56.973390 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.973370    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:56.976123 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.976107    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 21:58:56.986505 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.986476    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 21:58:56.996668 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:56.996645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6mm4\" (UniqueName: \"kubernetes.io/projected/b3a32913-e0bd-40ff-b8c9-e80a8a7adc58-kube-api-access-k6mm4\") pod \"network-check-target-k7cfk\" (UID: \"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58\") " pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:57.038916 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.038893    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-278n4\""
Sep 17 21:58:57.041987 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.041972    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djhj9"
Sep 17 21:58:57.046719 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.046703    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:58:57.048655 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.048639    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nbkcq\""
Sep 17 21:58:57.057626 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.057606    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xwm5p"
Sep 17 21:58:57.061333 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.061314    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c75b475d-d8cmf"]
Sep 17 21:58:57.064597 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.064579    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.074865 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.074843    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k8t4\" (UniqueName: \"kubernetes.io/projected/16d7f283-755e-49d9-944b-986fa7dec726-kube-api-access-8k8t4\") pod \"downloads-6594f4c75-wb4nh\" (UID: \"16d7f283-755e-49d9-944b-986fa7dec726\") " pod="openshift-console/downloads-6594f4c75-wb4nh"
Sep 17 21:58:57.077312 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.077238    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c75b475d-d8cmf"]
Sep 17 21:58:57.175955 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.175928    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcn7b\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-kube-api-access-hcn7b\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176069 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.175963    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-registry-certificates\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176069 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176023    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-registry-tls\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176138 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176084    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-ca-trust-extracted\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176171 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176156    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-bound-sa-token\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176205 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:57.176237 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176210    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-trusted-ca\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176294 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176236    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-image-registry-private-configuration\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176329 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176309    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:57.176359 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176337    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-installation-pull-secrets\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.176469 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.176455    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k8t4\" (UniqueName: \"kubernetes.io/projected/16d7f283-755e-49d9-944b-986fa7dec726-kube-api-access-8k8t4\") pod \"downloads-6594f4c75-wb4nh\" (UID: \"16d7f283-755e-49d9-944b-986fa7dec726\") " pod="openshift-console/downloads-6594f4c75-wb4nh"
Sep 17 21:58:57.178346 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.178324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddcb190d-cc99-43e6-b1ba-59a1b0450888-metrics-tls\") pod \"dns-default-gbdmh\" (UID: \"ddcb190d-cc99-43e6-b1ba-59a1b0450888\") " pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:57.178508 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.178492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"image-registry-85d57b5bc-rnmnl\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") " pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:57.183687 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.183667    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k8t4\" (UniqueName: \"kubernetes.io/projected/16d7f283-755e-49d9-944b-986fa7dec726-kube-api-access-8k8t4\") pod \"downloads-6594f4c75-wb4nh\" (UID: \"16d7f283-755e-49d9-944b-986fa7dec726\") " pod="openshift-console/downloads-6594f4c75-wb4nh"
Sep 17 21:58:57.208903 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.208881    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7dc29\""
Sep 17 21:58:57.216864 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.216842    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:58:57.268949 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.268801    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-wb4nh"
Sep 17 21:58:57.277392 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277366    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-trusted-ca\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.277502 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277413    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-image-registry-private-configuration\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.277502 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277442    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-installation-pull-secrets\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.277502 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcn7b\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-kube-api-access-hcn7b\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.277657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277509    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-registry-certificates\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.277657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-registry-tls\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.277657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-ca-trust-extracted\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.277657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.277625    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-bound-sa-token\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.278396 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.278270    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-ca-trust-extracted\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.278473 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.278403    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-trusted-ca\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.278473 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.278457    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-registry-certificates\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.280019 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.279997    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-installation-pull-secrets\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.280138 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.280123    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-image-registry-private-configuration\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.280662 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.280642    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-registry-tls\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.285336 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.285302    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcn7b\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-kube-api-access-hcn7b\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.285430 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.285414    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7dff0f76-948e-4e1a-bef1-f9013a4d8bc5-bound-sa-token\") pod \"image-registry-5c75b475d-d8cmf\" (UID: \"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5\") " pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.317304 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.317274    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djhj9"]
Sep 17 21:58:57.321051 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.320980    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k7cfk"]
Sep 17 21:58:57.325535 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:57.325486    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a32913_e0bd_40ff_b8c9_e80a8a7adc58.slice/crio-0237298e3848a1fcd31b6eefa9bd56487370a3808b141326ba9f7364c9de342b WatchSource:0}: Error finding container 0237298e3848a1fcd31b6eefa9bd56487370a3808b141326ba9f7364c9de342b: Status 404 returned error can't find the container with id 0237298e3848a1fcd31b6eefa9bd56487370a3808b141326ba9f7364c9de342b
Sep 17 21:58:57.346497 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.346473    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xwm5p"]
Sep 17 21:58:57.348534 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:57.348512    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8cccc3_163f_4c1e_af6e_6390d5a2a694.slice/crio-bf658db69d75d7de9e6018aa7919f477980ee3b13a31180e5d9611b312194c84 WatchSource:0}: Error finding container bf658db69d75d7de9e6018aa7919f477980ee3b13a31180e5d9611b312194c84: Status 404 returned error can't find the container with id bf658db69d75d7de9e6018aa7919f477980ee3b13a31180e5d9611b312194c84
Sep 17 21:58:57.378972 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.378953    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nrw4l\""
Sep 17 21:58:57.387712 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.387672    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:57.465178 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.465113    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djhj9" event={"ID":"9811d2bc-c727-42e3-a514-02ca5a24be2f","Type":"ContainerStarted","Data":"ff645d471d6f52882e9925e92fa5815cdf59d7b2dd9430813d81683cc6c588a5"}
Sep 17 21:58:57.466836 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.466763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xwm5p" event={"ID":"2e8cccc3-163f-4c1e-af6e-6390d5a2a694","Type":"ContainerStarted","Data":"bf658db69d75d7de9e6018aa7919f477980ee3b13a31180e5d9611b312194c84"}
Sep 17 21:58:57.468122 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.468090    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k7cfk" event={"ID":"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58","Type":"ContainerStarted","Data":"0237298e3848a1fcd31b6eefa9bd56487370a3808b141326ba9f7364c9de342b"}
Sep 17 21:58:57.468215 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.468137    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:57.475390 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.475372    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:57.483790 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.483771    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gbdmh"]
Sep 17 21:58:57.486416 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:57.486386    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddcb190d_cc99_43e6_b1ba_59a1b0450888.slice/crio-fb1d326b63c73336313f446c5be6842870f6d9932bc086f0407afd2d33ff24e6 WatchSource:0}: Error finding container fb1d326b63c73336313f446c5be6842870f6d9932bc086f0407afd2d33ff24e6: Status 404 returned error can't find the container with id fb1d326b63c73336313f446c5be6842870f6d9932bc086f0407afd2d33ff24e6
Sep 17 21:58:57.537048 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.537007    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-wb4nh"]
Sep 17 21:58:57.539978 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:57.539958    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d7f283_755e_49d9_944b_986fa7dec726.slice/crio-8dafc99f167f99e7202a14b5e79af196484c2e45ce503f2d7c407675178f107f WatchSource:0}: Error finding container 8dafc99f167f99e7202a14b5e79af196484c2e45ce503f2d7c407675178f107f: Status 404 returned error can't find the container with id 8dafc99f167f99e7202a14b5e79af196484c2e45ce503f2d7c407675178f107f
Sep 17 21:58:57.579093 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579070    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-bound-sa-token\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579175 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579101    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-installation-pull-secrets\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579175 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579139    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-image-registry-private-configuration\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579175 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579156    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkzhh\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-kube-api-access-vkzhh\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579361 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579187    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-trusted-ca\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579361 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579216    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8a6403-fbe9-483e-b22b-181fc07c76ad-ca-trust-extracted\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579361 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579274    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-certificates\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579361 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579306    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") pod \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\" (UID: \"9b8a6403-fbe9-483e-b22b-181fc07c76ad\") "
Sep 17 21:58:57.579699 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579665    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:58:57.579699 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579681    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b8a6403-fbe9-483e-b22b-181fc07c76ad-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:58:57.579822 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.579724    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:58:57.581306 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.581266    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:58:57.581306 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.581271    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-kube-api-access-vkzhh" (OuterVolumeSpecName: "kube-api-access-vkzhh") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "kube-api-access-vkzhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:57.581425 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.581327    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:57.581577 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.581560    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:58:57.581738 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.581718    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9b8a6403-fbe9-483e-b22b-181fc07c76ad" (UID: "9b8a6403-fbe9-483e-b22b-181fc07c76ad"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:58:57.679844 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679818    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-trusted-ca\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.679844 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679842    2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8a6403-fbe9-483e-b22b-181fc07c76ad-ca-trust-extracted\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.679994 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679851    2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-certificates\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.679994 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679860    2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-registry-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.679994 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679868    2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-bound-sa-token\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.679994 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679876    2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-installation-pull-secrets\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.679994 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679884    2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8a6403-fbe9-483e-b22b-181fc07c76ad-image-registry-private-configuration\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.679994 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.679892    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkzhh\" (UniqueName: \"kubernetes.io/projected/9b8a6403-fbe9-483e-b22b-181fc07c76ad-kube-api-access-vkzhh\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:58:57.683638 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:57.683616    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c75b475d-d8cmf"]
Sep 17 21:58:57.686074 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:58:57.686048    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dff0f76_948e_4e1a_bef1_f9013a4d8bc5.slice/crio-856214e2c98905fa8469d4c2b60279acc30341ed25b3afa2ea6bedd1c4e6e3b5 WatchSource:0}: Error finding container 856214e2c98905fa8469d4c2b60279acc30341ed25b3afa2ea6bedd1c4e6e3b5: Status 404 returned error can't find the container with id 856214e2c98905fa8469d4c2b60279acc30341ed25b3afa2ea6bedd1c4e6e3b5
Sep 17 21:58:58.475920 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.475880    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gbdmh" event={"ID":"ddcb190d-cc99-43e6-b1ba-59a1b0450888","Type":"ContainerStarted","Data":"fb1d326b63c73336313f446c5be6842870f6d9932bc086f0407afd2d33ff24e6"}
Sep 17 21:58:58.480283 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.479432    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf" event={"ID":"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5","Type":"ContainerStarted","Data":"466a1a12138388735231f6007d247b321b7d716b48db6446df75540d7ace253c"}
Sep 17 21:58:58.480283 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.479478    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf" event={"ID":"7dff0f76-948e-4e1a-bef1-f9013a4d8bc5","Type":"ContainerStarted","Data":"856214e2c98905fa8469d4c2b60279acc30341ed25b3afa2ea6bedd1c4e6e3b5"}
Sep 17 21:58:58.480283 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.480184    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:58:58.483130 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.483056    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-wb4nh" event={"ID":"16d7f283-755e-49d9-944b-986fa7dec726","Type":"ContainerStarted","Data":"8dafc99f167f99e7202a14b5e79af196484c2e45ce503f2d7c407675178f107f"}
Sep 17 21:58:58.483130 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.483080    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d57b5bc-rnmnl"
Sep 17 21:58:58.501916 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.500582    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf" podStartSLOduration=1.500563485 podStartE2EDuration="1.500563485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:58.499805227 +0000 UTC m=+69.314180606" watchObservedRunningTime="2026-09-17 21:58:58.500563485 +0000 UTC m=+69.314938857"
Sep 17 21:58:58.529889 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.529842    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85d57b5bc-rnmnl"]
Sep 17 21:58:58.533643 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:58.533579    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85d57b5bc-rnmnl"]
Sep 17 21:58:59.133177 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.133075    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b8a6403-fbe9-483e-b22b-181fc07c76ad" path="/var/lib/kubelet/pods/9b8a6403-fbe9-483e-b22b-181fc07c76ad/volumes"
Sep 17 21:58:59.492560 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.492445    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xwm5p" event={"ID":"2e8cccc3-163f-4c1e-af6e-6390d5a2a694","Type":"ContainerStarted","Data":"6e15432c3d871cbff89258941c2267cab5c364bc2ba483f2648d3ffec9d07310"}
Sep 17 21:58:59.492560 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.492511    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xwm5p" event={"ID":"2e8cccc3-163f-4c1e-af6e-6390d5a2a694","Type":"ContainerStarted","Data":"116955d9b1091fdc16176f85aa2680e39409307cf002fbc73389e4937701a536"}
Sep 17 21:58:59.507452 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.506435    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xwm5p" podStartSLOduration=67.316646697 podStartE2EDuration="1m8.506414985s" totalImagesPullingTime="1.189768288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:58:59.506026489 +0000 UTC m=+70.320401866" watchObservedRunningTime="2026-09-17 21:58:59.506414985 +0000 UTC m=+70.320790363"
Sep 17 21:58:59.940916 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.940891    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84dfd87dcd-px6vv"]
Sep 17 21:58:59.944064 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.944042    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:58:59.947781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.946719    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-msrvc\""
Sep 17 21:58:59.947781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.946923    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 21:58:59.947781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.947102    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 21:58:59.947781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.947280    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 21:58:59.947781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.947523    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 21:58:59.947781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.947744    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 21:58:59.955095 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:58:59.953947    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84dfd87dcd-px6vv"]
Sep 17 21:59:00.001783 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.001746    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-oauth-config\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.001906 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.001794    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqvjm\" (UniqueName: \"kubernetes.io/projected/304640c5-f190-4afa-b470-ad79e3958ca3-kube-api-access-sqvjm\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.001906 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.001858    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-serving-cert\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.001906 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.001892    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-console-config\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.002072 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.001934    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-oauth-serving-cert\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.002072 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.001977    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-service-ca\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.103185 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.103147    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-oauth-serving-cert\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.103346 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.103215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-service-ca\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.103346 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.103299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-oauth-config\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.103346 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.103330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqvjm\" (UniqueName: \"kubernetes.io/projected/304640c5-f190-4afa-b470-ad79e3958ca3-kube-api-access-sqvjm\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.103517 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.103358    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-serving-cert\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.103517 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.103382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-console-config\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.104143 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.104107    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-console-config\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.104806 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.104778    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-oauth-serving-cert\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.105003 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.104979    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-service-ca\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.108728 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.108701    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-oauth-config\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.112291 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.112267    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-serving-cert\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.113379 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.113356    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqvjm\" (UniqueName: \"kubernetes.io/projected/304640c5-f190-4afa-b470-ad79e3958ca3-kube-api-access-sqvjm\") pod \"console-84dfd87dcd-px6vv\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") " pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.270476 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.270398    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:00.499915 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.499879    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gbdmh" event={"ID":"ddcb190d-cc99-43e6-b1ba-59a1b0450888","Type":"ContainerStarted","Data":"874cedf633b5e734611813aa689e5f965531655bc13401d68fc19d3e2e08a527"}
Sep 17 21:59:00.500397 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.499937    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gbdmh" event={"ID":"ddcb190d-cc99-43e6-b1ba-59a1b0450888","Type":"ContainerStarted","Data":"b6a7734531b7297c479686ae2f11af378e97238d48afb34424b062bc27e106b1"}
Sep 17 21:59:00.517823 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.517720    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gbdmh" podStartSLOduration=33.074454857 podStartE2EDuration="35.517700112s" totalImagesPullingTime="2.443245255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:00.51530067 +0000 UTC m=+71.329676050" watchObservedRunningTime="2026-09-17 21:59:00.517700112 +0000 UTC m=+71.332075490"
Sep 17 21:59:00.555001 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:00.554924    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84dfd87dcd-px6vv"]
Sep 17 21:59:01.500628 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:59:01.500586    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod304640c5_f190_4afa_b470_ad79e3958ca3.slice/crio-f9b717c5edecb7448ad8e92fe8b3656d899de19a03353eb31debd5c238ee1f9d WatchSource:0}: Error finding container f9b717c5edecb7448ad8e92fe8b3656d899de19a03353eb31debd5c238ee1f9d: Status 404 returned error can't find the container with id f9b717c5edecb7448ad8e92fe8b3656d899de19a03353eb31debd5c238ee1f9d
Sep 17 21:59:01.503742 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:01.503649    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:59:02.499032 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.498998    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"]
Sep 17 21:59:02.502358 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.502336    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:02.505184 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.505162    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 17 21:59:02.505438 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.505397    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pklct\""
Sep 17 21:59:02.509765 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.509715    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"]
Sep 17 21:59:02.510138 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.510095    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dfd87dcd-px6vv" event={"ID":"304640c5-f190-4afa-b470-ad79e3958ca3","Type":"ContainerStarted","Data":"f9b717c5edecb7448ad8e92fe8b3656d899de19a03353eb31debd5c238ee1f9d"}
Sep 17 21:59:02.622925 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.622892    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c897a44-f7ce-48ec-8c6f-320fe94c7a2e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-zrl4z\" (UID: \"5c897a44-f7ce-48ec-8c6f-320fe94c7a2e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:02.724053 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:02.723974    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c897a44-f7ce-48ec-8c6f-320fe94c7a2e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-zrl4z\" (UID: \"5c897a44-f7ce-48ec-8c6f-320fe94c7a2e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:02.724218 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:59:02.724149    2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 21:59:02.724322 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:59:02.724260    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c897a44-f7ce-48ec-8c6f-320fe94c7a2e-tls-certificates podName:5c897a44-f7ce-48ec-8c6f-320fe94c7a2e nodeName:}" failed. No retries permitted until 2026-09-17 21:59:03.224219785 +0000 UTC m=+74.038595156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5c897a44-f7ce-48ec-8c6f-320fe94c7a2e-tls-certificates") pod "prometheus-operator-admission-webhook-5c6747dc7b-zrl4z" (UID: "5c897a44-f7ce-48ec-8c6f-320fe94c7a2e") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 21:59:03.228365 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.228330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c897a44-f7ce-48ec-8c6f-320fe94c7a2e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-zrl4z\" (UID: \"5c897a44-f7ce-48ec-8c6f-320fe94c7a2e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:03.231067 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.231038    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c897a44-f7ce-48ec-8c6f-320fe94c7a2e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-zrl4z\" (UID: \"5c897a44-f7ce-48ec-8c6f-320fe94c7a2e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:03.416041 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.416012    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:03.521866 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.521708    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djhj9" event={"ID":"9811d2bc-c727-42e3-a514-02ca5a24be2f","Type":"ContainerStarted","Data":"f5626215297af3dd859dd750edaa175187d3ca6a987b970f2140164fcc742a6f"}
Sep 17 21:59:03.528815 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.528780    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k7cfk" event={"ID":"b3a32913-e0bd-40ff-b8c9-e80a8a7adc58","Type":"ContainerStarted","Data":"fe91957b2f3162ed0008448231cd631754f0af08d97911322e7961bc819e193e"}
Sep 17 21:59:03.529310 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.529290    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:59:03.559183 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.558610    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k7cfk" podStartSLOduration=66.644283789 podStartE2EDuration="1m12.558589508s" totalImagesPullingTime="5.914305719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:03.558190997 +0000 UTC m=+74.372566374" watchObservedRunningTime="2026-09-17 21:59:03.558589508 +0000 UTC m=+74.372964887"
Sep 17 21:59:03.559183 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.558761    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-djhj9" podStartSLOduration=66.635551364 podStartE2EDuration="1m12.558753382s" totalImagesPullingTime="5.923202018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:03.538738772 +0000 UTC m=+74.353114149" watchObservedRunningTime="2026-09-17 21:59:03.558753382 +0000 UTC m=+74.373128760"
Sep 17 21:59:03.718631 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:03.718552    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"]
Sep 17 21:59:03.723641 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:59:03.722896    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c897a44_f7ce_48ec_8c6f_320fe94c7a2e.slice/crio-0148204b4a66c9bd0631d06b914cc0918275c75b6aaf030690fbb7b0847f3601 WatchSource:0}: Error finding container 0148204b4a66c9bd0631d06b914cc0918275c75b6aaf030690fbb7b0847f3601: Status 404 returned error can't find the container with id 0148204b4a66c9bd0631d06b914cc0918275c75b6aaf030690fbb7b0847f3601
Sep 17 21:59:04.536418 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:04.535888    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z" event={"ID":"5c897a44-f7ce-48ec-8c6f-320fe94c7a2e","Type":"ContainerStarted","Data":"0148204b4a66c9bd0631d06b914cc0918275c75b6aaf030690fbb7b0847f3601"}
Sep 17 21:59:08.554493 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:08.554446    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z" event={"ID":"5c897a44-f7ce-48ec-8c6f-320fe94c7a2e","Type":"ContainerStarted","Data":"b666d1e702174b1c69996f9e7aa0398e8d31f13de78ad6d596492047ab27182d"}
Sep 17 21:59:08.554918 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:08.554524    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:08.556856 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:08.556637    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dfd87dcd-px6vv" event={"ID":"304640c5-f190-4afa-b470-ad79e3958ca3","Type":"ContainerStarted","Data":"35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140"}
Sep 17 21:59:08.560136 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:08.560113    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z"
Sep 17 21:59:08.568743 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:08.568695    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-zrl4z" podStartSLOduration=2.592462623 podStartE2EDuration="6.568677008s" totalImagesPullingTime="3.976214385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:08.568149583 +0000 UTC m=+79.382524960" watchObservedRunningTime="2026-09-17 21:59:08.568677008 +0000 UTC m=+79.383052391"
Sep 17 21:59:08.601203 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:08.601153    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84dfd87dcd-px6vv" podStartSLOduration=3.339719852 podStartE2EDuration="9.601133952s" totalImagesPullingTime="6.2614141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:08.599591102 +0000 UTC m=+79.413966478" watchObservedRunningTime="2026-09-17 21:59:08.601133952 +0000 UTC m=+79.415509331"
Sep 17 21:59:10.271375 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:10.271337    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:10.271794 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:10.271424    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:10.272924 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:10.272898    2561 patch_prober.go:28] interesting pod/console-84dfd87dcd-px6vv container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" start-of-body=
Sep 17 21:59:10.273052 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:10.272943    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-84dfd87dcd-px6vv" podUID="304640c5-f190-4afa-b470-ad79e3958ca3" containerName="console" probeResult="failure" output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused"
Sep 17 21:59:11.513184 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:11.513147    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gbdmh"
Sep 17 21:59:16.588698 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.588641    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-wb4nh" event={"ID":"16d7f283-755e-49d9-944b-986fa7dec726","Type":"ContainerStarted","Data":"bb1e07d6010a29ea12f0c30a4f2358dfc6b673ce324b00d429738fba7abb2d23"}
Sep 17 21:59:16.589182 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.588767    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-wb4nh"
Sep 17 21:59:16.602796 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.602767    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-wb4nh"
Sep 17 21:59:16.608881 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.608823    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-wb4nh" podStartSLOduration=2.054240393 podStartE2EDuration="20.608789921s" totalImagesPullingTime="18.554549528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:16.607440322 +0000 UTC m=+87.421815698" watchObservedRunningTime="2026-09-17 21:59:16.608789921 +0000 UTC m=+87.423165312"
Sep 17 21:59:16.933746 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.933703    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b9sjh"]
Sep 17 21:59:16.939175 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.939148    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:16.941712 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.941685    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 21:59:16.941902 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.941887    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 21:59:16.942047 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.942032    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 21:59:16.942191 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.942174    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjmkc\""
Sep 17 21:59:16.942609 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.942589    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 21:59:16.942811 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.942794    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 21:59:16.942978 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:16.942961    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 21:59:17.037164 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037378 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037191    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lh2n\" (UniqueName: \"kubernetes.io/projected/773ee54f-7190-4537-93b2-777967b0b70c-kube-api-access-5lh2n\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037378 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037228    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-wtmp\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037378 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037315    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-textfile\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037378 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037342    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-sys\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037378 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037375    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-tls\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037404    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-root\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037431    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/773ee54f-7190-4537-93b2-777967b0b70c-metrics-client-ca\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.037657 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.037460    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138079 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138034    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-textfile\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138079 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138085    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-sys\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138368 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138117    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-tls\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138368 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138142    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-root\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138368 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138168    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/773ee54f-7190-4537-93b2-777967b0b70c-metrics-client-ca\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138368 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138194    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138368 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138232    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138368 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138291    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lh2n\" (UniqueName: \"kubernetes.io/projected/773ee54f-7190-4537-93b2-777967b0b70c-kube-api-access-5lh2n\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138368 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138317    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-wtmp\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138726 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-wtmp\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138825 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138803    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-textfile\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138890 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.138866    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-sys\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.138980 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:59:17.138962    2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 21:59:17.139039 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:59:17.139035    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-tls podName:773ee54f-7190-4537-93b2-777967b0b70c nodeName:}" failed. No retries permitted until 2026-09-17 21:59:17.639015311 +0000 UTC m=+88.453390684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-tls") pod "node-exporter-b9sjh" (UID: "773ee54f-7190-4537-93b2-777967b0b70c") : secret "node-exporter-tls" not found
Sep 17 21:59:17.139110 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.139080    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/773ee54f-7190-4537-93b2-777967b0b70c-root\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.139597 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.139567    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/773ee54f-7190-4537-93b2-777967b0b70c-metrics-client-ca\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.139990 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.139967    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.143125 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.142736    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.152560 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.152533    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lh2n\" (UniqueName: \"kubernetes.io/projected/773ee54f-7190-4537-93b2-777967b0b70c-kube-api-access-5lh2n\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.393561 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.393456    2561 patch_prober.go:28] interesting pod/image-registry-5c75b475d-d8cmf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 21:59:17.393561 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.393548    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf" podUID="7dff0f76-948e-4e1a-bef1-f9013a4d8bc5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 21:59:17.642183 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.642145    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-tls\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.644954 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.644890    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/773ee54f-7190-4537-93b2-777967b0b70c-node-exporter-tls\") pod \"node-exporter-b9sjh\" (UID: \"773ee54f-7190-4537-93b2-777967b0b70c\") " pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:17.853823 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:17.853785    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b9sjh"
Sep 17 21:59:18.598044 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:18.598002    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9sjh" event={"ID":"773ee54f-7190-4537-93b2-777967b0b70c","Type":"ContainerStarted","Data":"a49ab5fe2be3c9be0e74b2c0b562190a45333014ebee1f9d5e3590974cea537c"}
Sep 17 21:59:20.276486 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:20.276451    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:20.281092 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:20.281041    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:20.346766 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:20.346736    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84dfd87dcd-px6vv"]
Sep 17 21:59:20.505775 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:20.505748    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c75b475d-d8cmf"
Sep 17 21:59:20.608056 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:20.607972    2561 generic.go:356] "Generic (PLEG): container finished" podID="773ee54f-7190-4537-93b2-777967b0b70c" containerID="2de30e6459a8a695853647db6cf94d93d7919a3d81596128b6921b9722e4411a" exitCode=0
Sep 17 21:59:20.608218 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:20.608069    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9sjh" event={"ID":"773ee54f-7190-4537-93b2-777967b0b70c","Type":"ContainerDied","Data":"2de30e6459a8a695853647db6cf94d93d7919a3d81596128b6921b9722e4411a"}
Sep 17 21:59:21.618312 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.618270    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9sjh" event={"ID":"773ee54f-7190-4537-93b2-777967b0b70c","Type":"ContainerStarted","Data":"f5c9d881d31d1b6c1239e78550b4258d56c8f3435fed7bc7ab2d9a9a3daff231"}
Sep 17 21:59:21.618786 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.618348    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9sjh" event={"ID":"773ee54f-7190-4537-93b2-777967b0b70c","Type":"ContainerStarted","Data":"bc2d11880478007b272061066672b2c1ff7ffe1a5da28ff31faa32fef0ae04b7"}
Sep 17 21:59:21.652044 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.651988    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b9sjh" podStartSLOduration=3.627607285 podStartE2EDuration="5.651971931s" totalImagesPullingTime="1.93756493s" totalInitContainerRuntime="86.799716ms" isStatefulPod=true podCreationTimestamp="2026-09-17 21:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:21.650460454 +0000 UTC m=+92.464835833" watchObservedRunningTime="2026-09-17 21:59:21.651971931 +0000 UTC m=+92.466347309"
Sep 17 21:59:21.697492 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.697461    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"]
Sep 17 21:59:21.722220 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.722197    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"]
Sep 17 21:59:21.722389 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.722297    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:21.724531 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.724507    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 17 21:59:21.724663 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.724581    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7nn58\""
Sep 17 21:59:21.879121 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.879045    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bnt8\" (UniqueName: \"kubernetes.io/projected/4d0171e6-885b-472b-bc98-38054e993d70-kube-api-access-7bnt8\") pod \"monitoring-plugin-5984459ff8-tbfxh\" (UID: \"4d0171e6-885b-472b-bc98-38054e993d70\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:21.879121 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.879104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4d0171e6-885b-472b-bc98-38054e993d70-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-tbfxh\" (UID: \"4d0171e6-885b-472b-bc98-38054e993d70\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:21.980097 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.980058    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bnt8\" (UniqueName: \"kubernetes.io/projected/4d0171e6-885b-472b-bc98-38054e993d70-kube-api-access-7bnt8\") pod \"monitoring-plugin-5984459ff8-tbfxh\" (UID: \"4d0171e6-885b-472b-bc98-38054e993d70\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:21.980328 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.980106    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4d0171e6-885b-472b-bc98-38054e993d70-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-tbfxh\" (UID: \"4d0171e6-885b-472b-bc98-38054e993d70\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:21.982814 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.982786    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4d0171e6-885b-472b-bc98-38054e993d70-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-tbfxh\" (UID: \"4d0171e6-885b-472b-bc98-38054e993d70\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:21.987698 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:21.987670    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bnt8\" (UniqueName: \"kubernetes.io/projected/4d0171e6-885b-472b-bc98-38054e993d70-kube-api-access-7bnt8\") pod \"monitoring-plugin-5984459ff8-tbfxh\" (UID: \"4d0171e6-885b-472b-bc98-38054e993d70\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:22.034045 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:22.034020    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:22.333546 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:22.333514    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"]
Sep 17 21:59:22.336891 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:59:22.336866    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0171e6_885b_472b_bc98_38054e993d70.slice/crio-3fea096afd611182ab3c96fe70e5c875b4e70d97d7c7e256d741983d5cfbe0b6 WatchSource:0}: Error finding container 3fea096afd611182ab3c96fe70e5c875b4e70d97d7c7e256d741983d5cfbe0b6: Status 404 returned error can't find the container with id 3fea096afd611182ab3c96fe70e5c875b4e70d97d7c7e256d741983d5cfbe0b6
Sep 17 21:59:22.623158 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:22.623074    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh" event={"ID":"4d0171e6-885b-472b-bc98-38054e993d70","Type":"ContainerStarted","Data":"3fea096afd611182ab3c96fe70e5c875b4e70d97d7c7e256d741983d5cfbe0b6"}
Sep 17 21:59:23.347433 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.347399    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b6f64b67-49bvw"]
Sep 17 21:59:23.360748 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.360723    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b6f64b67-49bvw"]
Sep 17 21:59:23.360891 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.360838    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.373176 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.372881    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 21:59:23.491328 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.491285    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-service-ca\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.491494 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.491410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-config\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.491494 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.491459    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-trusted-ca-bundle\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.491615 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.491501    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j9sd\" (UniqueName: \"kubernetes.io/projected/508d56c8-f68a-4a29-b1ec-40605c8dc38b-kube-api-access-7j9sd\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.491615 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.491553    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-oauth-serving-cert\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.491721 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.491636    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-oauth-config\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.491780 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.491729    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-serving-cert\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.592366 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.592328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-service-ca\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.592860 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.592377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-config\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.592860 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.592410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-trusted-ca-bundle\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.592860 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.592441    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j9sd\" (UniqueName: \"kubernetes.io/projected/508d56c8-f68a-4a29-b1ec-40605c8dc38b-kube-api-access-7j9sd\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.592860 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.592482    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-oauth-serving-cert\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.592860 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.592538    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-oauth-config\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.592860 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.592585    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-serving-cert\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.593388 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.593230    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-config\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.593388 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.593331    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-trusted-ca-bundle\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.593692 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.593643    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-oauth-serving-cert\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.593888 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.593836    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-service-ca\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.596238 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.596193    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-serving-cert\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.596415 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.596394    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-oauth-config\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.601005 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.600940    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j9sd\" (UniqueName: \"kubernetes.io/projected/508d56c8-f68a-4a29-b1ec-40605c8dc38b-kube-api-access-7j9sd\") pod \"console-7b6f64b67-49bvw\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") " pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:23.678169 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:23.678135    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:24.136781 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.136702    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b6f64b67-49bvw"]
Sep 17 21:59:24.298514 ip-10-0-131-69 kubenswrapper[2561]: W0917 21:59:24.298461    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508d56c8_f68a_4a29_b1ec_40605c8dc38b.slice/crio-8d6194d33df57011018d95eb492f4b07305324ccddf0f79bc3af9fbf25283bd8 WatchSource:0}: Error finding container 8d6194d33df57011018d95eb492f4b07305324ccddf0f79bc3af9fbf25283bd8: Status 404 returned error can't find the container with id 8d6194d33df57011018d95eb492f4b07305324ccddf0f79bc3af9fbf25283bd8
Sep 17 21:59:24.633455 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.633423    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh" event={"ID":"4d0171e6-885b-472b-bc98-38054e993d70","Type":"ContainerStarted","Data":"c35ec329ad2f81906c69a037c0ddc52e10d4ebbe81ff27cd91aeab735f4f7a4e"}
Sep 17 21:59:24.633745 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.633695    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:24.635386 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.635355    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6f64b67-49bvw" event={"ID":"508d56c8-f68a-4a29-b1ec-40605c8dc38b","Type":"ContainerStarted","Data":"b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f"}
Sep 17 21:59:24.635497 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.635402    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6f64b67-49bvw" event={"ID":"508d56c8-f68a-4a29-b1ec-40605c8dc38b","Type":"ContainerStarted","Data":"8d6194d33df57011018d95eb492f4b07305324ccddf0f79bc3af9fbf25283bd8"}
Sep 17 21:59:24.639522 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.639466    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh"
Sep 17 21:59:24.648751 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.648712    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-tbfxh" podStartSLOduration=1.660031083 podStartE2EDuration="3.648694839s" totalImagesPullingTime="1.988663756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:24.647374661 +0000 UTC m=+95.461750040" watchObservedRunningTime="2026-09-17 21:59:24.648694839 +0000 UTC m=+95.463070217"
Sep 17 21:59:24.663320 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:24.663045    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b6f64b67-49bvw" podStartSLOduration=1.663030784 podStartE2EDuration="1.663030784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:59:24.662610167 +0000 UTC m=+95.476985545" watchObservedRunningTime="2026-09-17 21:59:24.663030784 +0000 UTC m=+95.477406161"
Sep 17 21:59:33.678650 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:33.678614    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:33.679099 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:33.678677    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:33.683356 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:33.683337    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:34.674086 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:34.674056    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 21:59:35.543091 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:35.543058    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k7cfk"
Sep 17 21:59:46.640226 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.640162    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84dfd87dcd-px6vv" podUID="304640c5-f190-4afa-b470-ad79e3958ca3" containerName="console" containerID="cri-o://35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140" gracePeriod=15
Sep 17 21:59:46.915913 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.915893    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:46.954941 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.954912    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-serving-cert\") pod \"304640c5-f190-4afa-b470-ad79e3958ca3\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") "
Sep 17 21:59:46.955090 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.954957    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-oauth-config\") pod \"304640c5-f190-4afa-b470-ad79e3958ca3\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") "
Sep 17 21:59:46.955090 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.954997    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-service-ca\") pod \"304640c5-f190-4afa-b470-ad79e3958ca3\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") "
Sep 17 21:59:46.955208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.955095    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-oauth-serving-cert\") pod \"304640c5-f190-4afa-b470-ad79e3958ca3\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") "
Sep 17 21:59:46.955208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.955133    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqvjm\" (UniqueName: \"kubernetes.io/projected/304640c5-f190-4afa-b470-ad79e3958ca3-kube-api-access-sqvjm\") pod \"304640c5-f190-4afa-b470-ad79e3958ca3\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") "
Sep 17 21:59:46.955208 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.955158    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-console-config\") pod \"304640c5-f190-4afa-b470-ad79e3958ca3\" (UID: \"304640c5-f190-4afa-b470-ad79e3958ca3\") "
Sep 17 21:59:46.955442 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.955411    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-service-ca" (OuterVolumeSpecName: "service-ca") pod "304640c5-f190-4afa-b470-ad79e3958ca3" (UID: "304640c5-f190-4afa-b470-ad79e3958ca3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:59:46.955527 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.955501    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-service-ca\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:59:46.955662 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.955536    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "304640c5-f190-4afa-b470-ad79e3958ca3" (UID: "304640c5-f190-4afa-b470-ad79e3958ca3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:59:46.955784 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.955702    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-console-config" (OuterVolumeSpecName: "console-config") pod "304640c5-f190-4afa-b470-ad79e3958ca3" (UID: "304640c5-f190-4afa-b470-ad79e3958ca3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:59:46.957377 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.957350    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "304640c5-f190-4afa-b470-ad79e3958ca3" (UID: "304640c5-f190-4afa-b470-ad79e3958ca3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:59:46.957463 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.957398    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "304640c5-f190-4afa-b470-ad79e3958ca3" (UID: "304640c5-f190-4afa-b470-ad79e3958ca3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:59:46.957463 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:46.957432    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/304640c5-f190-4afa-b470-ad79e3958ca3-kube-api-access-sqvjm" (OuterVolumeSpecName: "kube-api-access-sqvjm") pod "304640c5-f190-4afa-b470-ad79e3958ca3" (UID: "304640c5-f190-4afa-b470-ad79e3958ca3"). InnerVolumeSpecName "kube-api-access-sqvjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:59:47.056035 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.056003    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:59:47.056035 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.056029    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/304640c5-f190-4afa-b470-ad79e3958ca3-console-oauth-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:59:47.056035 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.056038    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-oauth-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:59:47.056233 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.056047    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqvjm\" (UniqueName: \"kubernetes.io/projected/304640c5-f190-4afa-b470-ad79e3958ca3-kube-api-access-sqvjm\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:59:47.056233 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.056057    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/304640c5-f190-4afa-b470-ad79e3958ca3-console-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 21:59:47.716024 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.715988    2561 generic.go:356] "Generic (PLEG): container finished" podID="304640c5-f190-4afa-b470-ad79e3958ca3" containerID="35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140" exitCode=0
Sep 17 21:59:47.716438 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.716030    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84dfd87dcd-px6vv"
Sep 17 21:59:47.716438 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.716070    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dfd87dcd-px6vv" event={"ID":"304640c5-f190-4afa-b470-ad79e3958ca3","Type":"ContainerDied","Data":"35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140"}
Sep 17 21:59:47.716438 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.716116    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84dfd87dcd-px6vv" event={"ID":"304640c5-f190-4afa-b470-ad79e3958ca3","Type":"ContainerDied","Data":"f9b717c5edecb7448ad8e92fe8b3656d899de19a03353eb31debd5c238ee1f9d"}
Sep 17 21:59:47.716438 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.716137    2561 scope.go:117] "RemoveContainer" containerID="35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140"
Sep 17 21:59:47.723677 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.723663    2561 scope.go:117] "RemoveContainer" containerID="35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140"
Sep 17 21:59:47.723921 ip-10-0-131-69 kubenswrapper[2561]: E0917 21:59:47.723906    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140\": container with ID starting with 35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140 not found: ID does not exist" containerID="35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140"
Sep 17 21:59:47.723958 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.723930    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140"} err="failed to get container status \"35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140\": rpc error: code = NotFound desc = could not find container \"35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140\": container with ID starting with 35a3e421ef83bbfff4da272574feb11d5004d35baf8d04d110a86035daee8140 not found: ID does not exist"
Sep 17 21:59:47.730771 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.730750    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84dfd87dcd-px6vv"]
Sep 17 21:59:47.732638 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:47.732613    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84dfd87dcd-px6vv"]
Sep 17 21:59:49.135228 ip-10-0-131-69 kubenswrapper[2561]: I0917 21:59:49.135195    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="304640c5-f190-4afa-b470-ad79e3958ca3" path="/var/lib/kubelet/pods/304640c5-f190-4afa-b470-ad79e3958ca3/volumes"
Sep 17 22:00:45.398729 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.398652    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d57d77b6c-glzmn"]
Sep 17 22:00:45.399116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.398949    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="304640c5-f190-4afa-b470-ad79e3958ca3" containerName="console"
Sep 17 22:00:45.399116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.398964    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="304640c5-f190-4afa-b470-ad79e3958ca3" containerName="console"
Sep 17 22:00:45.399116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.399028    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="304640c5-f190-4afa-b470-ad79e3958ca3" containerName="console"
Sep 17 22:00:45.405053 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.405026    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.413989 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.413964    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d57d77b6c-glzmn"]
Sep 17 22:00:45.553044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.553000    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-config\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.553044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.553047    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-trusted-ca-bundle\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.553263 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.553092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h27xr\" (UniqueName: \"kubernetes.io/projected/1b6b74a5-344c-487b-8c2b-a31a200d0a15-kube-api-access-h27xr\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.553263 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.553133    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-service-ca\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.553263 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.553210    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-oauth-config\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.553263 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.553242    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-serving-cert\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.553417 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.553293    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-oauth-serving-cert\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654015 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.653970    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-service-ca\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-oauth-config\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654042    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-serving-cert\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654071    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-oauth-serving-cert\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654089    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-config\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654105    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-trusted-ca-bundle\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654139    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h27xr\" (UniqueName: \"kubernetes.io/projected/1b6b74a5-344c-487b-8c2b-a31a200d0a15-kube-api-access-h27xr\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654856 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654816    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-service-ca\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.654977 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.654877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-oauth-serving-cert\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.655059 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.655036    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-trusted-ca-bundle\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.655125 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.655036    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-config\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.656980 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.656956    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-serving-cert\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.657116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.657093    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-oauth-config\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.662836 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.662813    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h27xr\" (UniqueName: \"kubernetes.io/projected/1b6b74a5-344c-487b-8c2b-a31a200d0a15-kube-api-access-h27xr\") pod \"console-7d57d77b6c-glzmn\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") " pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.714700 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.714673    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:00:45.983240 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:45.983201    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d57d77b6c-glzmn"]
Sep 17 22:00:45.985793 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:00:45.985769    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b6b74a5_344c_487b_8c2b_a31a200d0a15.slice/crio-db4c7f26c33f0e62f608aa03d92cb5822f9377c8f97b4b5f2bdc8fc851be2461 WatchSource:0}: Error finding container db4c7f26c33f0e62f608aa03d92cb5822f9377c8f97b4b5f2bdc8fc851be2461: Status 404 returned error can't find the container with id db4c7f26c33f0e62f608aa03d92cb5822f9377c8f97b4b5f2bdc8fc851be2461
Sep 17 22:00:46.924768 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:46.924732    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d57d77b6c-glzmn" event={"ID":"1b6b74a5-344c-487b-8c2b-a31a200d0a15","Type":"ContainerStarted","Data":"d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d"}
Sep 17 22:00:46.925136 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:46.924787    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d57d77b6c-glzmn" event={"ID":"1b6b74a5-344c-487b-8c2b-a31a200d0a15","Type":"ContainerStarted","Data":"db4c7f26c33f0e62f608aa03d92cb5822f9377c8f97b4b5f2bdc8fc851be2461"}
Sep 17 22:00:46.942043 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:46.941993    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d57d77b6c-glzmn" podStartSLOduration=1.9419781029999998 podStartE2EDuration="1.941978103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:00:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:00:46.940193949 +0000 UTC m=+177.754569325" watchObservedRunningTime="2026-09-17 22:00:46.941978103 +0000 UTC m=+177.756353480"
Sep 17 22:00:52.422263 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.422211    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d57d77b6c-glzmn"]
Sep 17 22:00:52.451695 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.451662    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69fbcf66d7-8jl8f"]
Sep 17 22:00:52.454886 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.454864    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.466084 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.466056    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69fbcf66d7-8jl8f"]
Sep 17 22:00:52.606145 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.606117    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-trusted-ca-bundle\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.606145 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.606146    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-oauth-serving-cert\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.606328 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.606171    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46x9n\" (UniqueName: \"kubernetes.io/projected/bdd8a22c-9971-43c4-a564-dee92b6d0318-kube-api-access-46x9n\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.606328 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.606232    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-oauth-config\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.606328 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.606276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-serving-cert\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.606328 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.606295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-service-ca\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.606328 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.606317    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-config\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.707483 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.707425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-oauth-config\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.707483 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.707461    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-serving-cert\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.707576 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.707488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-service-ca\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.707576 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.707511    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-config\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.707576 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.707540    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-trusted-ca-bundle\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.707576 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.707562    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-oauth-serving-cert\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.707751 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.707588    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46x9n\" (UniqueName: \"kubernetes.io/projected/bdd8a22c-9971-43c4-a564-dee92b6d0318-kube-api-access-46x9n\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.708239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.708217    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-service-ca\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.708362 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.708331    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-oauth-serving-cert\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.708402 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.708331    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-config\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.708621 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.708597    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-trusted-ca-bundle\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.709920 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.709895    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-serving-cert\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.709999 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.709956    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-oauth-config\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.714871 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.714850    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46x9n\" (UniqueName: \"kubernetes.io/projected/bdd8a22c-9971-43c4-a564-dee92b6d0318-kube-api-access-46x9n\") pod \"console-69fbcf66d7-8jl8f\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") " pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:52.763050 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:52.763023    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:00:53.030608 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:53.030529    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69fbcf66d7-8jl8f"]
Sep 17 22:00:53.033176 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:00:53.033144    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd8a22c_9971_43c4_a564_dee92b6d0318.slice/crio-34c27f709f832a6c38998a1ad623a785d93d7180d136b66725688fee57985070 WatchSource:0}: Error finding container 34c27f709f832a6c38998a1ad623a785d93d7180d136b66725688fee57985070: Status 404 returned error can't find the container with id 34c27f709f832a6c38998a1ad623a785d93d7180d136b66725688fee57985070
Sep 17 22:00:53.950316 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:53.950274    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fbcf66d7-8jl8f" event={"ID":"bdd8a22c-9971-43c4-a564-dee92b6d0318","Type":"ContainerStarted","Data":"19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737"}
Sep 17 22:00:53.950697 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:53.950330    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fbcf66d7-8jl8f" event={"ID":"bdd8a22c-9971-43c4-a564-dee92b6d0318","Type":"ContainerStarted","Data":"34c27f709f832a6c38998a1ad623a785d93d7180d136b66725688fee57985070"}
Sep 17 22:00:53.966053 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:53.966000    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69fbcf66d7-8jl8f" podStartSLOduration=1.96598622 podStartE2EDuration="1.96598622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:00:53.965837854 +0000 UTC m=+184.780213230" watchObservedRunningTime="2026-09-17 22:00:53.96598622 +0000 UTC m=+184.780361597"
Sep 17 22:00:55.714907 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:00:55.714875    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:01:02.764086 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:02.764051    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:01:02.764577 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:02.764136    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:01:02.768798 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:02.768775    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:01:02.987024 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:02.986998    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:01:03.029126 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:03.029048    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b6f64b67-49bvw"]
Sep 17 22:01:17.440411 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.440369    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d57d77b6c-glzmn" podUID="1b6b74a5-344c-487b-8c2b-a31a200d0a15" containerName="console" containerID="cri-o://d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d" gracePeriod=15
Sep 17 22:01:17.674215 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.674192    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:01:17.785601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785533    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-trusted-ca-bundle\") pod \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") "
Sep 17 22:01:17.785601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785562    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h27xr\" (UniqueName: \"kubernetes.io/projected/1b6b74a5-344c-487b-8c2b-a31a200d0a15-kube-api-access-h27xr\") pod \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") "
Sep 17 22:01:17.785770 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785697    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-oauth-serving-cert\") pod \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") "
Sep 17 22:01:17.785770 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785729    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-serving-cert\") pod \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") "
Sep 17 22:01:17.785770 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785754    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-config\") pod \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") "
Sep 17 22:01:17.785897 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785772    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-oauth-config\") pod \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") "
Sep 17 22:01:17.785897 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785794    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-service-ca\") pod \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\" (UID: \"1b6b74a5-344c-487b-8c2b-a31a200d0a15\") "
Sep 17 22:01:17.785996 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.785976    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1b6b74a5-344c-487b-8c2b-a31a200d0a15" (UID: "1b6b74a5-344c-487b-8c2b-a31a200d0a15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:17.786056 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.786032    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1b6b74a5-344c-487b-8c2b-a31a200d0a15" (UID: "1b6b74a5-344c-487b-8c2b-a31a200d0a15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:17.786286 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.786233    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-config" (OuterVolumeSpecName: "console-config") pod "1b6b74a5-344c-487b-8c2b-a31a200d0a15" (UID: "1b6b74a5-344c-487b-8c2b-a31a200d0a15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:17.786286 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.786236    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-service-ca" (OuterVolumeSpecName: "service-ca") pod "1b6b74a5-344c-487b-8c2b-a31a200d0a15" (UID: "1b6b74a5-344c-487b-8c2b-a31a200d0a15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:17.787798 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.787777    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1b6b74a5-344c-487b-8c2b-a31a200d0a15" (UID: "1b6b74a5-344c-487b-8c2b-a31a200d0a15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:01:17.787877 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.787854    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b6b74a5-344c-487b-8c2b-a31a200d0a15-kube-api-access-h27xr" (OuterVolumeSpecName: "kube-api-access-h27xr") pod "1b6b74a5-344c-487b-8c2b-a31a200d0a15" (UID: "1b6b74a5-344c-487b-8c2b-a31a200d0a15"). InnerVolumeSpecName "kube-api-access-h27xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:01:17.787923 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.787854    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1b6b74a5-344c-487b-8c2b-a31a200d0a15" (UID: "1b6b74a5-344c-487b-8c2b-a31a200d0a15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:01:17.886967 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.886933    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-oauth-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:17.886967 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.886959    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:17.886967 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.886968    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:17.887167 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.886978    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b6b74a5-344c-487b-8c2b-a31a200d0a15-console-oauth-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:17.887167 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.886986    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-service-ca\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:17.887167 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.886994    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b6b74a5-344c-487b-8c2b-a31a200d0a15-trusted-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:17.887167 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:17.887002    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h27xr\" (UniqueName: \"kubernetes.io/projected/1b6b74a5-344c-487b-8c2b-a31a200d0a15-kube-api-access-h27xr\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:18.036256 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.036162    2561 generic.go:356] "Generic (PLEG): container finished" podID="1b6b74a5-344c-487b-8c2b-a31a200d0a15" containerID="d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d" exitCode=0
Sep 17 22:01:18.036256 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.036201    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d57d77b6c-glzmn" event={"ID":"1b6b74a5-344c-487b-8c2b-a31a200d0a15","Type":"ContainerDied","Data":"d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d"}
Sep 17 22:01:18.036256 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.036220    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d57d77b6c-glzmn"
Sep 17 22:01:18.036256 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.036236    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d57d77b6c-glzmn" event={"ID":"1b6b74a5-344c-487b-8c2b-a31a200d0a15","Type":"ContainerDied","Data":"db4c7f26c33f0e62f608aa03d92cb5822f9377c8f97b4b5f2bdc8fc851be2461"}
Sep 17 22:01:18.036522 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.036270    2561 scope.go:117] "RemoveContainer" containerID="d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d"
Sep 17 22:01:18.044756 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.044739    2561 scope.go:117] "RemoveContainer" containerID="d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d"
Sep 17 22:01:18.045030 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:01:18.045012    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d\": container with ID starting with d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d not found: ID does not exist" containerID="d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d"
Sep 17 22:01:18.045076 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.045038    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d"} err="failed to get container status \"d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d\": rpc error: code = NotFound desc = could not find container \"d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d\": container with ID starting with d12eac46fe0b935a48fae8936e75c355d75f11448f6582cc386d97244658ae5d not found: ID does not exist"
Sep 17 22:01:18.055585 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.055562    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d57d77b6c-glzmn"]
Sep 17 22:01:18.057279 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:18.057261    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d57d77b6c-glzmn"]
Sep 17 22:01:19.131544 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:19.131508    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b6b74a5-344c-487b-8c2b-a31a200d0a15" path="/var/lib/kubelet/pods/1b6b74a5-344c-487b-8c2b-a31a200d0a15/volumes"
Sep 17 22:01:28.048214 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.048151    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b6f64b67-49bvw" podUID="508d56c8-f68a-4a29-b1ec-40605c8dc38b" containerName="console" containerID="cri-o://b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f" gracePeriod=15
Sep 17 22:01:28.291188 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.291164    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 22:01:28.357517 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357490    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-trusted-ca-bundle\") pod \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") "
Sep 17 22:01:28.357691 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357528    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-oauth-config\") pod \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") "
Sep 17 22:01:28.357691 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357548    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-service-ca\") pod \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") "
Sep 17 22:01:28.357691 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357563    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-oauth-serving-cert\") pod \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") "
Sep 17 22:01:28.357824 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357767    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j9sd\" (UniqueName: \"kubernetes.io/projected/508d56c8-f68a-4a29-b1ec-40605c8dc38b-kube-api-access-7j9sd\") pod \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") "
Sep 17 22:01:28.357824 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357810    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-serving-cert\") pod \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") "
Sep 17 22:01:28.357932 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357838    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-config\") pod \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\" (UID: \"508d56c8-f68a-4a29-b1ec-40605c8dc38b\") "
Sep 17 22:01:28.357932 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357893    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-service-ca" (OuterVolumeSpecName: "service-ca") pod "508d56c8-f68a-4a29-b1ec-40605c8dc38b" (UID: "508d56c8-f68a-4a29-b1ec-40605c8dc38b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:28.358034 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357941    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "508d56c8-f68a-4a29-b1ec-40605c8dc38b" (UID: "508d56c8-f68a-4a29-b1ec-40605c8dc38b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:28.358034 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.357986    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "508d56c8-f68a-4a29-b1ec-40605c8dc38b" (UID: "508d56c8-f68a-4a29-b1ec-40605c8dc38b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:28.358136 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.358087    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-trusted-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:28.358220 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.358205    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-service-ca\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:28.358289 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.358225    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-oauth-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:28.358289 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.358280    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-config" (OuterVolumeSpecName: "console-config") pod "508d56c8-f68a-4a29-b1ec-40605c8dc38b" (UID: "508d56c8-f68a-4a29-b1ec-40605c8dc38b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:01:28.359622 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.359597    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/508d56c8-f68a-4a29-b1ec-40605c8dc38b-kube-api-access-7j9sd" (OuterVolumeSpecName: "kube-api-access-7j9sd") pod "508d56c8-f68a-4a29-b1ec-40605c8dc38b" (UID: "508d56c8-f68a-4a29-b1ec-40605c8dc38b"). InnerVolumeSpecName "kube-api-access-7j9sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:01:28.360174 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.360153    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "508d56c8-f68a-4a29-b1ec-40605c8dc38b" (UID: "508d56c8-f68a-4a29-b1ec-40605c8dc38b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:01:28.360216 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.360151    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "508d56c8-f68a-4a29-b1ec-40605c8dc38b" (UID: "508d56c8-f68a-4a29-b1ec-40605c8dc38b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:01:28.458675 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.458649    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j9sd\" (UniqueName: \"kubernetes.io/projected/508d56c8-f68a-4a29-b1ec-40605c8dc38b-kube-api-access-7j9sd\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:28.458675 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.458673    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:28.458823 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.458682    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:28.458823 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:28.458691    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/508d56c8-f68a-4a29-b1ec-40605c8dc38b-console-oauth-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:01:29.075290 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.075237    2561 generic.go:356] "Generic (PLEG): container finished" podID="508d56c8-f68a-4a29-b1ec-40605c8dc38b" containerID="b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f" exitCode=0
Sep 17 22:01:29.075673 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.075311    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6f64b67-49bvw"
Sep 17 22:01:29.075673 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.075323    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6f64b67-49bvw" event={"ID":"508d56c8-f68a-4a29-b1ec-40605c8dc38b","Type":"ContainerDied","Data":"b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f"}
Sep 17 22:01:29.075673 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.075378    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6f64b67-49bvw" event={"ID":"508d56c8-f68a-4a29-b1ec-40605c8dc38b","Type":"ContainerDied","Data":"8d6194d33df57011018d95eb492f4b07305324ccddf0f79bc3af9fbf25283bd8"}
Sep 17 22:01:29.075673 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.075397    2561 scope.go:117] "RemoveContainer" containerID="b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f"
Sep 17 22:01:29.083302 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.083282    2561 scope.go:117] "RemoveContainer" containerID="b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f"
Sep 17 22:01:29.083569 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:01:29.083552    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f\": container with ID starting with b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f not found: ID does not exist" containerID="b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f"
Sep 17 22:01:29.083614 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.083579    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f"} err="failed to get container status \"b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f\": rpc error: code = NotFound desc = could not find container \"b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f\": container with ID starting with b2276b67cfc5025dae2264aecfa81101f3714ccfbee6cd566bed5a37b116ab9f not found: ID does not exist"
Sep 17 22:01:29.094329 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.094303    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b6f64b67-49bvw"]
Sep 17 22:01:29.099103 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.099081    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b6f64b67-49bvw"]
Sep 17 22:01:29.131087 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:29.131063    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="508d56c8-f68a-4a29-b1ec-40605c8dc38b" path="/var/lib/kubelet/pods/508d56c8-f68a-4a29-b1ec-40605c8dc38b/volumes"
Sep 17 22:01:39.932445 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.932413    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"]
Sep 17 22:01:39.932908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.932672    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b6b74a5-344c-487b-8c2b-a31a200d0a15" containerName="console"
Sep 17 22:01:39.932908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.932683    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b6b74a5-344c-487b-8c2b-a31a200d0a15" containerName="console"
Sep 17 22:01:39.932908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.932694    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="508d56c8-f68a-4a29-b1ec-40605c8dc38b" containerName="console"
Sep 17 22:01:39.932908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.932700    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="508d56c8-f68a-4a29-b1ec-40605c8dc38b" containerName="console"
Sep 17 22:01:39.932908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.932746    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b6b74a5-344c-487b-8c2b-a31a200d0a15" containerName="console"
Sep 17 22:01:39.932908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.932754    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="508d56c8-f68a-4a29-b1ec-40605c8dc38b" containerName="console"
Sep 17 22:01:39.935427 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.935412    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:39.937446 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.937421    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 22:01:39.937553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.937478    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 22:01:39.937553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.937499    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 22:01:39.938544 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.938525    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 17 22:01:39.938611 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.938566    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6rbs4\""
Sep 17 22:01:39.944315 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:39.944293    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"]
Sep 17 22:01:40.030400 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.030365    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c0e6938-c6da-4524-bce1-7d02f19b048b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp\" (UID: \"8c0e6938-c6da-4524-bce1-7d02f19b048b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:40.030517 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.030435    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5652v\" (UniqueName: \"kubernetes.io/projected/8c0e6938-c6da-4524-bce1-7d02f19b048b-kube-api-access-5652v\") pod \"managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp\" (UID: \"8c0e6938-c6da-4524-bce1-7d02f19b048b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:40.130968 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.130938    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5652v\" (UniqueName: \"kubernetes.io/projected/8c0e6938-c6da-4524-bce1-7d02f19b048b-kube-api-access-5652v\") pod \"managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp\" (UID: \"8c0e6938-c6da-4524-bce1-7d02f19b048b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:40.131114 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.130982    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c0e6938-c6da-4524-bce1-7d02f19b048b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp\" (UID: \"8c0e6938-c6da-4524-bce1-7d02f19b048b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:40.133411 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.133394    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c0e6938-c6da-4524-bce1-7d02f19b048b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp\" (UID: \"8c0e6938-c6da-4524-bce1-7d02f19b048b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:40.138765 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.138741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5652v\" (UniqueName: \"kubernetes.io/projected/8c0e6938-c6da-4524-bce1-7d02f19b048b-kube-api-access-5652v\") pod \"managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp\" (UID: \"8c0e6938-c6da-4524-bce1-7d02f19b048b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:40.256502 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.256428    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"
Sep 17 22:01:40.522080 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:40.521994    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp"]
Sep 17 22:01:40.526428 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:01:40.526401    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c0e6938_c6da_4524_bce1_7d02f19b048b.slice/crio-87a3d96040e314b07947ef0845c78c9d4a46f084ef96a0d0007eecd45e67cab3 WatchSource:0}: Error finding container 87a3d96040e314b07947ef0845c78c9d4a46f084ef96a0d0007eecd45e67cab3: Status 404 returned error can't find the container with id 87a3d96040e314b07947ef0845c78c9d4a46f084ef96a0d0007eecd45e67cab3
Sep 17 22:01:41.115956 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:41.115924    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp" event={"ID":"8c0e6938-c6da-4524-bce1-7d02f19b048b","Type":"ContainerStarted","Data":"87a3d96040e314b07947ef0845c78c9d4a46f084ef96a0d0007eecd45e67cab3"}
Sep 17 22:01:44.128977 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:44.128942    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp" event={"ID":"8c0e6938-c6da-4524-bce1-7d02f19b048b","Type":"ContainerStarted","Data":"0d58f2a32d981e96abe15af8d9d24627cd95482ed4e9ecb2076964f135455a9f"}
Sep 17 22:01:44.142795 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:01:44.142749    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c84dfc9f6-7l8mp" podStartSLOduration=2.583276103 podStartE2EDuration="5.142733011s" totalImagesPullingTime="2.559456908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:01:44.141737078 +0000 UTC m=+234.956112454" watchObservedRunningTime="2026-09-17 22:01:44.142733011 +0000 UTC m=+234.957108389"
Sep 17 22:02:09.554262 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.554219    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"]
Sep 17 22:02:09.557445 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.557429    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.559805 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.559778    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 22:02:09.559935 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.559850    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 22:02:09.560343 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.560326    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-66gbz\""
Sep 17 22:02:09.565530 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.565510    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"]
Sep 17 22:02:09.633042 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.633006    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.633042 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.633044    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4lp\" (UniqueName: \"kubernetes.io/projected/a593b113-62d0-440e-a872-d7c1d6dfea6c-kube-api-access-zr4lp\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.633270 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.633100    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.733893 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.733863    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.734011 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.733920    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.734113 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.734084    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4lp\" (UniqueName: \"kubernetes.io/projected/a593b113-62d0-440e-a872-d7c1d6dfea6c-kube-api-access-zr4lp\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.734344 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.734324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.734426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.734339    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.741769 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.741751    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4lp\" (UniqueName: \"kubernetes.io/projected/a593b113-62d0-440e-a872-d7c1d6dfea6c-kube-api-access-zr4lp\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:09.867446 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:09.867362    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:10.133886 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:10.133799    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"]
Sep 17 22:02:10.136346 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:02:10.136317    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda593b113_62d0_440e_a872_d7c1d6dfea6c.slice/crio-5b1a24a0f761b75a3036acfa52f9a30f6ba5fd46a38c2aa1b81c7f517d82813c WatchSource:0}: Error finding container 5b1a24a0f761b75a3036acfa52f9a30f6ba5fd46a38c2aa1b81c7f517d82813c: Status 404 returned error can't find the container with id 5b1a24a0f761b75a3036acfa52f9a30f6ba5fd46a38c2aa1b81c7f517d82813c
Sep 17 22:02:10.215041 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:10.215008    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts" event={"ID":"a593b113-62d0-440e-a872-d7c1d6dfea6c","Type":"ContainerStarted","Data":"5b1a24a0f761b75a3036acfa52f9a30f6ba5fd46a38c2aa1b81c7f517d82813c"}
Sep 17 22:02:17.242883 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:17.242843    2561 generic.go:356] "Generic (PLEG): container finished" podID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerID="f9ac1c7e7884a447241073818996af17f66cd78bab99fd118d536c8cc8680601" exitCode=0
Sep 17 22:02:17.243330 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:17.242933    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts" event={"ID":"a593b113-62d0-440e-a872-d7c1d6dfea6c","Type":"ContainerDied","Data":"f9ac1c7e7884a447241073818996af17f66cd78bab99fd118d536c8cc8680601"}
Sep 17 22:02:20.259155 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:20.259116    2561 generic.go:356] "Generic (PLEG): container finished" podID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerID="bed94092a351ab417e7c1875dd25d2628c4d3bf16ce663ce21d48aa58f20b559" exitCode=0
Sep 17 22:02:20.259553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:20.259196    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts" event={"ID":"a593b113-62d0-440e-a872-d7c1d6dfea6c","Type":"ContainerDied","Data":"bed94092a351ab417e7c1875dd25d2628c4d3bf16ce663ce21d48aa58f20b559"}
Sep 17 22:02:28.299343 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:28.299231    2561 generic.go:356] "Generic (PLEG): container finished" podID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerID="9230e02685811d870673f54922e49f564f69c0347737c268d1a1edaa316f3f3f" exitCode=0
Sep 17 22:02:28.299343 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:28.299280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts" event={"ID":"a593b113-62d0-440e-a872-d7c1d6dfea6c","Type":"ContainerDied","Data":"9230e02685811d870673f54922e49f564f69c0347737c268d1a1edaa316f3f3f"}
Sep 17 22:02:29.432412 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.432387    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:29.496728 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.496703    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-bundle\") pod \"a593b113-62d0-440e-a872-d7c1d6dfea6c\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") "
Sep 17 22:02:29.496836 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.496734    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr4lp\" (UniqueName: \"kubernetes.io/projected/a593b113-62d0-440e-a872-d7c1d6dfea6c-kube-api-access-zr4lp\") pod \"a593b113-62d0-440e-a872-d7c1d6dfea6c\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") "
Sep 17 22:02:29.496836 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.496767    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-util\") pod \"a593b113-62d0-440e-a872-d7c1d6dfea6c\" (UID: \"a593b113-62d0-440e-a872-d7c1d6dfea6c\") "
Sep 17 22:02:29.497343 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.497315    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-bundle" (OuterVolumeSpecName: "bundle") pod "a593b113-62d0-440e-a872-d7c1d6dfea6c" (UID: "a593b113-62d0-440e-a872-d7c1d6dfea6c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:02:29.498774 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.498740    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a593b113-62d0-440e-a872-d7c1d6dfea6c-kube-api-access-zr4lp" (OuterVolumeSpecName: "kube-api-access-zr4lp") pod "a593b113-62d0-440e-a872-d7c1d6dfea6c" (UID: "a593b113-62d0-440e-a872-d7c1d6dfea6c"). InnerVolumeSpecName "kube-api-access-zr4lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:02:29.501022 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.501004    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-util" (OuterVolumeSpecName: "util") pod "a593b113-62d0-440e-a872-d7c1d6dfea6c" (UID: "a593b113-62d0-440e-a872-d7c1d6dfea6c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 22:02:29.597660 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.597605    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:02:29.597660 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.597625    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr4lp\" (UniqueName: \"kubernetes.io/projected/a593b113-62d0-440e-a872-d7c1d6dfea6c-kube-api-access-zr4lp\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:02:29.597660 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:29.597636    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a593b113-62d0-440e-a872-d7c1d6dfea6c-util\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:02:30.308521 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:30.308498    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts"
Sep 17 22:02:30.308698 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:30.308493    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aj9mts" event={"ID":"a593b113-62d0-440e-a872-d7c1d6dfea6c","Type":"ContainerDied","Data":"5b1a24a0f761b75a3036acfa52f9a30f6ba5fd46a38c2aa1b81c7f517d82813c"}
Sep 17 22:02:30.308698 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:30.308615    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b1a24a0f761b75a3036acfa52f9a30f6ba5fd46a38c2aa1b81c7f517d82813c"
Sep 17 22:02:36.645820 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.645785    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"]
Sep 17 22:02:36.646205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.646021    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerName="util"
Sep 17 22:02:36.646205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.646032    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerName="util"
Sep 17 22:02:36.646205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.646044    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerName="pull"
Sep 17 22:02:36.646205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.646049    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerName="pull"
Sep 17 22:02:36.646205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.646054    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerName="extract"
Sep 17 22:02:36.646205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.646059    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerName="extract"
Sep 17 22:02:36.646205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.646100    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a593b113-62d0-440e-a872-d7c1d6dfea6c" containerName="extract"
Sep 17 22:02:36.681942 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.681909    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"]
Sep 17 22:02:36.682078 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.682003    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:36.684963 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.684941    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-wnwhj\""
Sep 17 22:02:36.685389 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.685372    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 22:02:36.685616 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.685600    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 17 22:02:36.685884 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.685870    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 22:02:36.846158 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.846123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/78bda971-6a7c-4da2-91ae-29c1c447500d-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cbvxx\" (UID: \"78bda971-6a7c-4da2-91ae-29c1c447500d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:36.846342 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.846172    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rkwg\" (UniqueName: \"kubernetes.io/projected/78bda971-6a7c-4da2-91ae-29c1c447500d-kube-api-access-6rkwg\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cbvxx\" (UID: \"78bda971-6a7c-4da2-91ae-29c1c447500d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:36.947474 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.947451    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/78bda971-6a7c-4da2-91ae-29c1c447500d-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cbvxx\" (UID: \"78bda971-6a7c-4da2-91ae-29c1c447500d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:36.947583 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.947492    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rkwg\" (UniqueName: \"kubernetes.io/projected/78bda971-6a7c-4da2-91ae-29c1c447500d-kube-api-access-6rkwg\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cbvxx\" (UID: \"78bda971-6a7c-4da2-91ae-29c1c447500d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:36.949710 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.949689    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/78bda971-6a7c-4da2-91ae-29c1c447500d-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cbvxx\" (UID: \"78bda971-6a7c-4da2-91ae-29c1c447500d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:36.957118 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.957092    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rkwg\" (UniqueName: \"kubernetes.io/projected/78bda971-6a7c-4da2-91ae-29c1c447500d-kube-api-access-6rkwg\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cbvxx\" (UID: \"78bda971-6a7c-4da2-91ae-29c1c447500d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:36.991902 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:36.991878    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:37.280011 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:37.279988    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"]
Sep 17 22:02:37.282333 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:02:37.282306    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bda971_6a7c_4da2_91ae_29c1c447500d.slice/crio-c94ea5febfe586d46317f45d43ad6e6d4d2f96755014c49e180d300375e9d64a WatchSource:0}: Error finding container c94ea5febfe586d46317f45d43ad6e6d4d2f96755014c49e180d300375e9d64a: Status 404 returned error can't find the container with id c94ea5febfe586d46317f45d43ad6e6d4d2f96755014c49e180d300375e9d64a
Sep 17 22:02:37.333696 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:37.333669    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx" event={"ID":"78bda971-6a7c-4da2-91ae-29c1c447500d","Type":"ContainerStarted","Data":"c94ea5febfe586d46317f45d43ad6e6d4d2f96755014c49e180d300375e9d64a"}
Sep 17 22:02:41.353557 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.353516    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx" event={"ID":"78bda971-6a7c-4da2-91ae-29c1c447500d","Type":"ContainerStarted","Data":"56f80d4c9cb5c255b9de5fa58ddbd6239f0e03c89df409175e72797438de921f"}
Sep 17 22:02:41.353996 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.353769    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:02:41.375517 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.375480    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx" podStartSLOduration=1.809604693 podStartE2EDuration="5.375449827s" totalImagesPullingTime="3.565845134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:02:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:02:41.374599131 +0000 UTC m=+292.188974521" watchObservedRunningTime="2026-09-17 22:02:41.375449827 +0000 UTC m=+292.189825198"
Sep 17 22:02:41.551581 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.551551    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-t7b8j"]
Sep 17 22:02:41.574368 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.574343    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-t7b8j"]
Sep 17 22:02:41.574513 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.574445    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.576926 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.576906    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 22:02:41.576926 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.576922    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-22m6g\""
Sep 17 22:02:41.577086 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.576907    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 17 22:02:41.683566 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.683542    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.683566 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.683577    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d5e0f2ce-a333-4e6b-9507-adba52f336f2-cabundle0\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.683783 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.683604    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v58tq\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-kube-api-access-v58tq\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.752056 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.752029    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"]
Sep 17 22:02:41.771096 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.771073    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"]
Sep 17 22:02:41.771234 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.771202    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.775286 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.775263    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 17 22:02:41.784805 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.784784    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d5e0f2ce-a333-4e6b-9507-adba52f336f2-cabundle0\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.784928 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.784816    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v58tq\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-kube-api-access-v58tq\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.784988 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.784927    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.785040 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.785011    2561 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Sep 17 22:02:41.785040 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.785023    2561 secret.go:281] references non-existent secret key: ca.crt
Sep 17 22:02:41.785040 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.785030    2561 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 22:02:41.785040 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.785040    2561 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-t7b8j: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 22:02:41.785225 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.785081    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates podName:d5e0f2ce-a333-4e6b-9507-adba52f336f2 nodeName:}" failed. No retries permitted until 2026-09-17 22:02:42.285067167 +0000 UTC m=+293.099442522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates") pod "keda-operator-579c8958f7-t7b8j" (UID: "d5e0f2ce-a333-4e6b-9507-adba52f336f2") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 22:02:41.785502 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.785482    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d5e0f2ce-a333-4e6b-9507-adba52f336f2-cabundle0\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.795077 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.795056    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v58tq\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-kube-api-access-v58tq\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:41.885727 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.885686    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.885901 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.885737    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtnpd\" (UniqueName: \"kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-kube-api-access-qtnpd\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.885901 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.885780    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-certificates\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.987150 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.987069    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-certificates\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.987321 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.987162    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.987321 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.987187    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtnpd\" (UniqueName: \"kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-kube-api-access-qtnpd\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.987321 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.987203    2561 secret.go:281] references non-existent secret key: tls.crt
Sep 17 22:02:41.987321 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.987221    2561 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 17 22:02:41.987321 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.987241    2561 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg: references non-existent secret key: tls.crt
Sep 17 22:02:41.987321 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:41.987319    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-certificates podName:25df8b7c-5b61-4a83-89c1-7ff8f6702dd0 nodeName:}" failed. No retries permitted until 2026-09-17 22:02:42.487301565 +0000 UTC m=+293.301676925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-certificates") pod "keda-metrics-apiserver-77857b7b98-x2rvg" (UID: "25df8b7c-5b61-4a83-89c1-7ff8f6702dd0") : references non-existent secret key: tls.crt
Sep 17 22:02:41.987622 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.987573    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:41.997160 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:41.997141    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtnpd\" (UniqueName: \"kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-kube-api-access-qtnpd\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:42.289641 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.289558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:42.289801 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:42.289667    2561 secret.go:281] references non-existent secret key: ca.crt
Sep 17 22:02:42.289801 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:42.289680    2561 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 22:02:42.289801 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:42.289689    2561 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-t7b8j: references non-existent secret key: ca.crt
Sep 17 22:02:42.289801 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:02:42.289733    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates podName:d5e0f2ce-a333-4e6b-9507-adba52f336f2 nodeName:}" failed. No retries permitted until 2026-09-17 22:02:43.289721325 +0000 UTC m=+294.104096681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates") pod "keda-operator-579c8958f7-t7b8j" (UID: "d5e0f2ce-a333-4e6b-9507-adba52f336f2") : references non-existent secret key: ca.crt
Sep 17 22:02:42.491402 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.491234    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-certificates\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:42.494339 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.494317    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/25df8b7c-5b61-4a83-89c1-7ff8f6702dd0-certificates\") pod \"keda-metrics-apiserver-77857b7b98-x2rvg\" (UID: \"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:42.526548 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.526515    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-rgqb6"]
Sep 17 22:02:42.529237 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.529222    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.531623 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.531601    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 17 22:02:42.538804 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.538782    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-rgqb6"]
Sep 17 22:02:42.683700 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.683672    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:42.693342 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.693320    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcqsz\" (UniqueName: \"kubernetes.io/projected/55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958-kube-api-access-mcqsz\") pod \"keda-admission-6c4d864b86-rgqb6\" (UID: \"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958\") " pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.693433 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.693354    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958-certificates\") pod \"keda-admission-6c4d864b86-rgqb6\" (UID: \"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958\") " pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.794734 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.794704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcqsz\" (UniqueName: \"kubernetes.io/projected/55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958-kube-api-access-mcqsz\") pod \"keda-admission-6c4d864b86-rgqb6\" (UID: \"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958\") " pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.794919 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.794740    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958-certificates\") pod \"keda-admission-6c4d864b86-rgqb6\" (UID: \"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958\") " pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.797199 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.797177    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958-certificates\") pod \"keda-admission-6c4d864b86-rgqb6\" (UID: \"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958\") " pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.806081 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.806057    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcqsz\" (UniqueName: \"kubernetes.io/projected/55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958-kube-api-access-mcqsz\") pod \"keda-admission-6c4d864b86-rgqb6\" (UID: \"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958\") " pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.840038 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.840009    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:42.951032 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:42.950998    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"]
Sep 17 22:02:42.953146 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:02:42.953120    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25df8b7c_5b61_4a83_89c1_7ff8f6702dd0.slice/crio-fe724bdfad0ac2e2eb69ccb06782904687788434fc31bfa54c035bedca8529d7 WatchSource:0}: Error finding container fe724bdfad0ac2e2eb69ccb06782904687788434fc31bfa54c035bedca8529d7: Status 404 returned error can't find the container with id fe724bdfad0ac2e2eb69ccb06782904687788434fc31bfa54c035bedca8529d7
Sep 17 22:02:43.110011 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:43.109978    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-rgqb6"]
Sep 17 22:02:43.114198 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:02:43.114159    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dd8e6d_4a5d_4cb2_824d_1a8ad84b3958.slice/crio-914fcfd3431cca2ea0deceaee55c056e93331352a1817f099eb5144c4e4560dd WatchSource:0}: Error finding container 914fcfd3431cca2ea0deceaee55c056e93331352a1817f099eb5144c4e4560dd: Status 404 returned error can't find the container with id 914fcfd3431cca2ea0deceaee55c056e93331352a1817f099eb5144c4e4560dd
Sep 17 22:02:43.300538 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:43.300457    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:43.302805 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:43.302774    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d5e0f2ce-a333-4e6b-9507-adba52f336f2-certificates\") pod \"keda-operator-579c8958f7-t7b8j\" (UID: \"d5e0f2ce-a333-4e6b-9507-adba52f336f2\") " pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:43.362551 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:43.362514    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-rgqb6" event={"ID":"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958","Type":"ContainerStarted","Data":"914fcfd3431cca2ea0deceaee55c056e93331352a1817f099eb5144c4e4560dd"}
Sep 17 22:02:43.363590 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:43.363566    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg" event={"ID":"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0","Type":"ContainerStarted","Data":"fe724bdfad0ac2e2eb69ccb06782904687788434fc31bfa54c035bedca8529d7"}
Sep 17 22:02:43.386304 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:43.386279    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:43.669788 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:02:43.669762    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e0f2ce_a333_4e6b_9507_adba52f336f2.slice/crio-f2091aa6e1cbce951432dd11192eb9f56f41ee013a14e5a172ad0156c79a5977 WatchSource:0}: Error finding container f2091aa6e1cbce951432dd11192eb9f56f41ee013a14e5a172ad0156c79a5977: Status 404 returned error can't find the container with id f2091aa6e1cbce951432dd11192eb9f56f41ee013a14e5a172ad0156c79a5977
Sep 17 22:02:43.680832 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:43.680805    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-t7b8j"]
Sep 17 22:02:44.369558 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:44.369507    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-t7b8j" event={"ID":"d5e0f2ce-a333-4e6b-9507-adba52f336f2","Type":"ContainerStarted","Data":"f2091aa6e1cbce951432dd11192eb9f56f41ee013a14e5a172ad0156c79a5977"}
Sep 17 22:02:45.376075 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:45.376040    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-rgqb6" event={"ID":"55dd8e6d-4a5d-4cb2-824d-1a8ad84b3958","Type":"ContainerStarted","Data":"24645d519c28ebdcd7b013705eccf4144f916cf9c3f38ef92e7c4f3f0b7d5ee7"}
Sep 17 22:02:45.376494 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:45.376198    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:02:45.391437 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:45.391359    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-rgqb6" podStartSLOduration=1.996170615 podStartE2EDuration="3.391343172s" totalImagesPullingTime="1.395172557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:02:45.391002665 +0000 UTC m=+296.205378041" watchObservedRunningTime="2026-09-17 22:02:45.391343172 +0000 UTC m=+296.205718548"
Sep 17 22:02:47.386333 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:47.386287    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-t7b8j" event={"ID":"d5e0f2ce-a333-4e6b-9507-adba52f336f2","Type":"ContainerStarted","Data":"1039d6f3782aa4221788bd44121ff1eea741bb3212ef74140ec8dfc85b30716f"}
Sep 17 22:02:47.386688 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:47.386391    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:02:47.404056 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:47.404010    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-t7b8j" podStartSLOduration=2.915088763 podStartE2EDuration="6.403996692s" totalImagesPullingTime="3.488907929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:02:47.402232523 +0000 UTC m=+298.216607901" watchObservedRunningTime="2026-09-17 22:02:47.403996692 +0000 UTC m=+298.218372069"
Sep 17 22:02:51.066885 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:51.066860    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:02:51.067861 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:51.067842    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:02:51.072225 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:51.072207    2561 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 22:02:54.416376 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:54.416339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg" event={"ID":"25df8b7c-5b61-4a83-89c1-7ff8f6702dd0","Type":"ContainerStarted","Data":"7a465944eb3a97531198f91d13d0b9244a888f53894df1635518d0ac2bb61fa3"}
Sep 17 22:02:54.416780 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:54.416489    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:02:54.431730 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:02:54.431657    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg" podStartSLOduration=2.244285034 podStartE2EDuration="13.431642374s" totalImagesPullingTime="11.18735734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:02:54.430812706 +0000 UTC m=+305.245188080" watchObservedRunningTime="2026-09-17 22:02:54.431642374 +0000 UTC m=+305.246017751"
Sep 17 22:03:02.360081 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:02.360054    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cbvxx"
Sep 17 22:03:05.424966 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:05.424932    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-x2rvg"
Sep 17 22:03:06.384126 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:06.384100    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-rgqb6"
Sep 17 22:03:08.392169 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:08.392143    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-t7b8j"
Sep 17 22:03:57.369796 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.369711    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-hxqdc"]
Sep 17 22:03:57.373010 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.372988    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.375243 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.375226    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 22:03:57.376161 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.376144    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-mfzp6\""
Sep 17 22:03:57.376242 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.376144    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 17 22:03:57.376242 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.376146    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 22:03:57.381235 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.381211    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-hxqdc"]
Sep 17 22:03:57.452066 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.452042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c037a4f8-7dbc-420f-a585-538ee81c5078-data\") pod \"seaweedfs-64568bcd49-hxqdc\" (UID: \"c037a4f8-7dbc-420f-a585-538ee81c5078\") " pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.452220 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.452085    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggn54\" (UniqueName: \"kubernetes.io/projected/c037a4f8-7dbc-420f-a585-538ee81c5078-kube-api-access-ggn54\") pod \"seaweedfs-64568bcd49-hxqdc\" (UID: \"c037a4f8-7dbc-420f-a585-538ee81c5078\") " pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.553308 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.553275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c037a4f8-7dbc-420f-a585-538ee81c5078-data\") pod \"seaweedfs-64568bcd49-hxqdc\" (UID: \"c037a4f8-7dbc-420f-a585-538ee81c5078\") " pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.553435 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.553330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggn54\" (UniqueName: \"kubernetes.io/projected/c037a4f8-7dbc-420f-a585-538ee81c5078-kube-api-access-ggn54\") pod \"seaweedfs-64568bcd49-hxqdc\" (UID: \"c037a4f8-7dbc-420f-a585-538ee81c5078\") " pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.553709 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.553691    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c037a4f8-7dbc-420f-a585-538ee81c5078-data\") pod \"seaweedfs-64568bcd49-hxqdc\" (UID: \"c037a4f8-7dbc-420f-a585-538ee81c5078\") " pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.562066 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.562044    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggn54\" (UniqueName: \"kubernetes.io/projected/c037a4f8-7dbc-420f-a585-538ee81c5078-kube-api-access-ggn54\") pod \"seaweedfs-64568bcd49-hxqdc\" (UID: \"c037a4f8-7dbc-420f-a585-538ee81c5078\") " pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.682614 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.682590    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:03:57.957299 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.957273    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-hxqdc"]
Sep 17 22:03:57.961860 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:57.961836    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 22:03:58.675236 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:03:58.675199    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-hxqdc" event={"ID":"c037a4f8-7dbc-420f-a585-538ee81c5078","Type":"ContainerStarted","Data":"45c82f4dea2d2ee11ac43d022488c6178886e13899ccf0400f7e33a61946b5c9"}
Sep 17 22:04:01.688555 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:01.688521    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-hxqdc" event={"ID":"c037a4f8-7dbc-420f-a585-538ee81c5078","Type":"ContainerStarted","Data":"2c1b4b11f3721f9a5132bf376cc89bda1f9cbf6a46eababa41e63498a4fd7cca"}
Sep 17 22:04:01.688973 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:01.688666    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:04:01.703911 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:01.703868    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-hxqdc" podStartSLOduration=1.800471988 podStartE2EDuration="4.70385406s" totalImagesPullingTime="2.903382072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:04:01.702380925 +0000 UTC m=+372.516756301" watchObservedRunningTime="2026-09-17 22:04:01.70385406 +0000 UTC m=+372.518229436"
Sep 17 22:04:07.695397 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:07.695365    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-hxqdc"
Sep 17 22:04:10.767816 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.767784    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5688df7978-9ctpx"]
Sep 17 22:04:10.772101 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.772085    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.783666 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.783645    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5688df7978-9ctpx"]
Sep 17 22:04:10.847965 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.847935    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-oauth-config\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.847965 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.847964    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-service-ca\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.848149 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.847983    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-serving-cert\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.848149 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.848000    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f88p2\" (UniqueName: \"kubernetes.io/projected/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-kube-api-access-f88p2\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.848149 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.848028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-oauth-serving-cert\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.848149 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.848043    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-config\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.848149 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.848063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-trusted-ca-bundle\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.948695 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.948661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-oauth-config\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.948812 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.948699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-service-ca\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.948812 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.948726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-serving-cert\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.948812 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.948747    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f88p2\" (UniqueName: \"kubernetes.io/projected/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-kube-api-access-f88p2\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.948812 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.948781    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-oauth-serving-cert\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.948812 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.948808    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-config\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.949097 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.948846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-trusted-ca-bundle\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.949561 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.949530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-service-ca\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.949679 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.949530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-config\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.949679 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.949632    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-oauth-serving-cert\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.949757 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.949714    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-trusted-ca-bundle\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.951277 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.951233    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-oauth-config\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.951277 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.951240    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-console-serving-cert\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:10.956353 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:10.956329    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f88p2\" (UniqueName: \"kubernetes.io/projected/c030a11f-4607-4e1f-9a28-c4ddf4c2bb44-kube-api-access-f88p2\") pod \"console-5688df7978-9ctpx\" (UID: \"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44\") " pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:11.080838 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:11.080762    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:11.349678 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:11.349654    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5688df7978-9ctpx"]
Sep 17 22:04:11.351623 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:04:11.351586    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc030a11f_4607_4e1f_9a28_c4ddf4c2bb44.slice/crio-b62193541126fb1164b7b1e80da5545d611121e3c5c292324418c5029e71c50c WatchSource:0}: Error finding container b62193541126fb1164b7b1e80da5545d611121e3c5c292324418c5029e71c50c: Status 404 returned error can't find the container with id b62193541126fb1164b7b1e80da5545d611121e3c5c292324418c5029e71c50c
Sep 17 22:04:11.730422 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:11.730380    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5688df7978-9ctpx" event={"ID":"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44","Type":"ContainerStarted","Data":"43efc7fdf45e5ff653d0074925889942c480c5f8f8232e39d16fb2996c5dd066"}
Sep 17 22:04:11.730594 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:11.730437    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5688df7978-9ctpx" event={"ID":"c030a11f-4607-4e1f-9a28-c4ddf4c2bb44","Type":"ContainerStarted","Data":"b62193541126fb1164b7b1e80da5545d611121e3c5c292324418c5029e71c50c"}
Sep 17 22:04:11.747132 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:11.747089    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5688df7978-9ctpx" podStartSLOduration=1.74707054 podStartE2EDuration="1.74707054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:04:11.745110612 +0000 UTC m=+382.559485993" watchObservedRunningTime="2026-09-17 22:04:11.74707054 +0000 UTC m=+382.561445917"
Sep 17 22:04:21.081665 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:21.081633    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:21.082101 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:21.081711    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:21.086420 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:21.086401    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:21.773772 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:21.773741    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5688df7978-9ctpx"
Sep 17 22:04:21.816632 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:21.816590    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69fbcf66d7-8jl8f"]
Sep 17 22:04:46.836599 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:46.836539    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69fbcf66d7-8jl8f" podUID="bdd8a22c-9971-43c4-a564-dee92b6d0318" containerName="console" containerID="cri-o://19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737" gracePeriod=15
Sep 17 22:04:47.073150 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.073122    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:04:47.118477 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118417    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-oauth-serving-cert\") pod \"bdd8a22c-9971-43c4-a564-dee92b6d0318\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") "
Sep 17 22:04:47.118477 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118451    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-service-ca\") pod \"bdd8a22c-9971-43c4-a564-dee92b6d0318\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") "
Sep 17 22:04:47.118477 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118473    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-config\") pod \"bdd8a22c-9971-43c4-a564-dee92b6d0318\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") "
Sep 17 22:04:47.118731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118493    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-oauth-config\") pod \"bdd8a22c-9971-43c4-a564-dee92b6d0318\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") "
Sep 17 22:04:47.118731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118524    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46x9n\" (UniqueName: \"kubernetes.io/projected/bdd8a22c-9971-43c4-a564-dee92b6d0318-kube-api-access-46x9n\") pod \"bdd8a22c-9971-43c4-a564-dee92b6d0318\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") "
Sep 17 22:04:47.118731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118556    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-serving-cert\") pod \"bdd8a22c-9971-43c4-a564-dee92b6d0318\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") "
Sep 17 22:04:47.118731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118577    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-trusted-ca-bundle\") pod \"bdd8a22c-9971-43c4-a564-dee92b6d0318\" (UID: \"bdd8a22c-9971-43c4-a564-dee92b6d0318\") "
Sep 17 22:04:47.118917 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118774    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bdd8a22c-9971-43c4-a564-dee92b6d0318" (UID: "bdd8a22c-9971-43c4-a564-dee92b6d0318"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:04:47.118917 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118811    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-service-ca" (OuterVolumeSpecName: "service-ca") pod "bdd8a22c-9971-43c4-a564-dee92b6d0318" (UID: "bdd8a22c-9971-43c4-a564-dee92b6d0318"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:04:47.119050 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.118882    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-config" (OuterVolumeSpecName: "console-config") pod "bdd8a22c-9971-43c4-a564-dee92b6d0318" (UID: "bdd8a22c-9971-43c4-a564-dee92b6d0318"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:04:47.119140 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.119114    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bdd8a22c-9971-43c4-a564-dee92b6d0318" (UID: "bdd8a22c-9971-43c4-a564-dee92b6d0318"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:04:47.120643 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.120601    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bdd8a22c-9971-43c4-a564-dee92b6d0318" (UID: "bdd8a22c-9971-43c4-a564-dee92b6d0318"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:04:47.120747 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.120642    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdd8a22c-9971-43c4-a564-dee92b6d0318-kube-api-access-46x9n" (OuterVolumeSpecName: "kube-api-access-46x9n") pod "bdd8a22c-9971-43c4-a564-dee92b6d0318" (UID: "bdd8a22c-9971-43c4-a564-dee92b6d0318"). InnerVolumeSpecName "kube-api-access-46x9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:04:47.120747 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.120696    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bdd8a22c-9971-43c4-a564-dee92b6d0318" (UID: "bdd8a22c-9971-43c4-a564-dee92b6d0318"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:04:47.219318 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.219297    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-oauth-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:04:47.219318 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.219318    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46x9n\" (UniqueName: \"kubernetes.io/projected/bdd8a22c-9971-43c4-a564-dee92b6d0318-kube-api-access-46x9n\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:04:47.219464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.219328    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:04:47.219464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.219337    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-trusted-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:04:47.219464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.219345    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-oauth-serving-cert\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:04:47.219464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.219354    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-service-ca\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:04:47.219464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.219362    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdd8a22c-9971-43c4-a564-dee92b6d0318-console-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:04:47.870825 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.870791    2561 generic.go:356] "Generic (PLEG): container finished" podID="bdd8a22c-9971-43c4-a564-dee92b6d0318" containerID="19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737" exitCode=0
Sep 17 22:04:47.871314 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.870839    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fbcf66d7-8jl8f"
Sep 17 22:04:47.871314 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.870846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fbcf66d7-8jl8f" event={"ID":"bdd8a22c-9971-43c4-a564-dee92b6d0318","Type":"ContainerDied","Data":"19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737"}
Sep 17 22:04:47.871314 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.870884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fbcf66d7-8jl8f" event={"ID":"bdd8a22c-9971-43c4-a564-dee92b6d0318","Type":"ContainerDied","Data":"34c27f709f832a6c38998a1ad623a785d93d7180d136b66725688fee57985070"}
Sep 17 22:04:47.871314 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.870903    2561 scope.go:117] "RemoveContainer" containerID="19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737"
Sep 17 22:04:47.879110 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.879090    2561 scope.go:117] "RemoveContainer" containerID="19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737"
Sep 17 22:04:47.879383 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:04:47.879362    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737\": container with ID starting with 19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737 not found: ID does not exist" containerID="19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737"
Sep 17 22:04:47.879435 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.879391    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737"} err="failed to get container status \"19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737\": rpc error: code = NotFound desc = could not find container \"19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737\": container with ID starting with 19355b8958f14445b84ab3f4b46913e3426d59c6a1211eb7066e8cbd32d3b737 not found: ID does not exist"
Sep 17 22:04:47.885949 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.885926    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69fbcf66d7-8jl8f"]
Sep 17 22:04:47.889629 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:47.889607    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69fbcf66d7-8jl8f"]
Sep 17 22:04:49.132540 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:04:49.132510    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdd8a22c-9971-43c4-a564-dee92b6d0318" path="/var/lib/kubelet/pods/bdd8a22c-9971-43c4-a564-dee92b6d0318/volumes"
Sep 17 22:05:13.470429 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.470395    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-9rct5"]
Sep 17 22:05:13.470970 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.470860    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdd8a22c-9971-43c4-a564-dee92b6d0318" containerName="console"
Sep 17 22:05:13.470970 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.470879    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdd8a22c-9971-43c4-a564-dee92b6d0318" containerName="console"
Sep 17 22:05:13.470970 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.470956    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdd8a22c-9971-43c4-a564-dee92b6d0318" containerName="console"
Sep 17 22:05:13.473960 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.473937    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:13.476408 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.476387    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 17 22:05:13.476576 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.476398    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-l55nf\""
Sep 17 22:05:13.485239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.483307    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-9rct5"]
Sep 17 22:05:13.487828 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.487809    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-cs95q"]
Sep 17 22:05:13.490884 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.490868    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.493113 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.493092    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 17 22:05:13.493318 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.493110    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 17 22:05:13.493662 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.493201    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-hjklg\""
Sep 17 22:05:13.503266 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.503228    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-cs95q"]
Sep 17 22:05:13.604274 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.604225    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d5hl\" (UniqueName: \"kubernetes.io/projected/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-kube-api-access-8d5hl\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.604452 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.604288    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/648b64cb-e97e-4e27-a477-65a0177896b1-tls-certs\") pod \"model-serving-api-7b954ffc95-9rct5\" (UID: \"648b64cb-e97e-4e27-a477-65a0177896b1\") " pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:13.604452 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.604320    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm9nt\" (UniqueName: \"kubernetes.io/projected/648b64cb-e97e-4e27-a477-65a0177896b1-kube-api-access-hm9nt\") pod \"model-serving-api-7b954ffc95-9rct5\" (UID: \"648b64cb-e97e-4e27-a477-65a0177896b1\") " pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:13.604452 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.604358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-cert\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.604452 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.604440    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-metrics-certs\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.705359 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.705327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-cert\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.705514 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.705384    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-metrics-certs\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.705514 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.705438    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d5hl\" (UniqueName: \"kubernetes.io/projected/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-kube-api-access-8d5hl\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.705514 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.705466    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/648b64cb-e97e-4e27-a477-65a0177896b1-tls-certs\") pod \"model-serving-api-7b954ffc95-9rct5\" (UID: \"648b64cb-e97e-4e27-a477-65a0177896b1\") " pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:13.705514 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:05:13.705476    2561 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 17 22:05:13.705514 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.705491    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm9nt\" (UniqueName: \"kubernetes.io/projected/648b64cb-e97e-4e27-a477-65a0177896b1-kube-api-access-hm9nt\") pod \"model-serving-api-7b954ffc95-9rct5\" (UID: \"648b64cb-e97e-4e27-a477-65a0177896b1\") " pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:13.705819 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:05:13.705548    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-cert podName:6e732cb9-e64e-475f-bd69-d78cb6ddcaa0 nodeName:}" failed. No retries permitted until 2026-09-17 22:05:14.205525728 +0000 UTC m=+445.019901095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-cert") pod "odh-model-controller-7b966d7984-cs95q" (UID: "6e732cb9-e64e-475f-bd69-d78cb6ddcaa0") : secret "odh-model-controller-webhook-cert" not found
Sep 17 22:05:13.705819 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:05:13.705646    2561 secret.go:189] Couldn't get secret kserve/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found
Sep 17 22:05:13.705819 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:05:13.705701    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-metrics-certs podName:6e732cb9-e64e-475f-bd69-d78cb6ddcaa0 nodeName:}" failed. No retries permitted until 2026-09-17 22:05:14.205685273 +0000 UTC m=+445.020060629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-metrics-certs") pod "odh-model-controller-7b966d7984-cs95q" (UID: "6e732cb9-e64e-475f-bd69-d78cb6ddcaa0") : secret "odh-model-controller-metrics-tls" not found
Sep 17 22:05:13.708101 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.708075    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/648b64cb-e97e-4e27-a477-65a0177896b1-tls-certs\") pod \"model-serving-api-7b954ffc95-9rct5\" (UID: \"648b64cb-e97e-4e27-a477-65a0177896b1\") " pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:13.713103 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.713077    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d5hl\" (UniqueName: \"kubernetes.io/projected/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-kube-api-access-8d5hl\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:13.713693 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.713674    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm9nt\" (UniqueName: \"kubernetes.io/projected/648b64cb-e97e-4e27-a477-65a0177896b1-kube-api-access-hm9nt\") pod \"model-serving-api-7b954ffc95-9rct5\" (UID: \"648b64cb-e97e-4e27-a477-65a0177896b1\") " pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:13.788169 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:13.788092    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:14.061968 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.061895    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-9rct5"]
Sep 17 22:05:14.065674 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:05:14.065647    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648b64cb_e97e_4e27_a477_65a0177896b1.slice/crio-1827c353e3503768d6ed4a5f28f8f4f4a9169e2549890c860358b07d7468bffd WatchSource:0}: Error finding container 1827c353e3503768d6ed4a5f28f8f4f4a9169e2549890c860358b07d7468bffd: Status 404 returned error can't find the container with id 1827c353e3503768d6ed4a5f28f8f4f4a9169e2549890c860358b07d7468bffd
Sep 17 22:05:14.208980 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.208947    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-cert\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:14.209107 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.209005    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-metrics-certs\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:14.211233 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.211211    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-cert\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:14.211340 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.211292    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e732cb9-e64e-475f-bd69-d78cb6ddcaa0-metrics-certs\") pod \"odh-model-controller-7b966d7984-cs95q\" (UID: \"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0\") " pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:14.407616 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.407594    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:14.675594 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.675522    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-cs95q"]
Sep 17 22:05:14.678758 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:05:14.678729    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e732cb9_e64e_475f_bd69_d78cb6ddcaa0.slice/crio-d41e78ba0ad7ea8d30693639f3ff50385dac69765de2a42713452a4cf0d118c0 WatchSource:0}: Error finding container d41e78ba0ad7ea8d30693639f3ff50385dac69765de2a42713452a4cf0d118c0: Status 404 returned error can't find the container with id d41e78ba0ad7ea8d30693639f3ff50385dac69765de2a42713452a4cf0d118c0
Sep 17 22:05:14.985058 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.984967    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-9rct5" event={"ID":"648b64cb-e97e-4e27-a477-65a0177896b1","Type":"ContainerStarted","Data":"1827c353e3503768d6ed4a5f28f8f4f4a9169e2549890c860358b07d7468bffd"}
Sep 17 22:05:14.986641 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:14.986608    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-cs95q" event={"ID":"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0","Type":"ContainerStarted","Data":"d41e78ba0ad7ea8d30693639f3ff50385dac69765de2a42713452a4cf0d118c0"}
Sep 17 22:05:18.001719 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:18.001680    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-9rct5" event={"ID":"648b64cb-e97e-4e27-a477-65a0177896b1","Type":"ContainerStarted","Data":"1d1985acd2ca599a5ecb5dc40b8698e0b75c665e1eef05b58d1edbcf6fd5f294"}
Sep 17 22:05:18.002143 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:18.001785    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:05:18.003427 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:18.003391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-cs95q" event={"ID":"6e732cb9-e64e-475f-bd69-d78cb6ddcaa0","Type":"ContainerStarted","Data":"1109a9d222d06d87e22f8fc85be6cc944ae85b04bdf9b967e524cd8d83458771"}
Sep 17 22:05:18.003538 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:18.003478    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:18.018598 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:18.018557    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-9rct5" podStartSLOduration=1.288994948 podStartE2EDuration="5.018544995s" totalImagesPullingTime="3.729550047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:05:18.017981939 +0000 UTC m=+448.832357316" watchObservedRunningTime="2026-09-17 22:05:18.018544995 +0000 UTC m=+448.832920441"
Sep 17 22:05:18.035380 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:18.035339    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-cs95q" podStartSLOduration=1.861461261 podStartE2EDuration="5.035324383s" totalImagesPullingTime="3.173863122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:05:18.03357508 +0000 UTC m=+448.847950456" watchObservedRunningTime="2026-09-17 22:05:18.035324383 +0000 UTC m=+448.849699760"
Sep 17 22:05:29.010168 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:29.010136    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-cs95q"
Sep 17 22:05:29.012181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:05:29.012156    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-9rct5"
Sep 17 22:07:01.799056 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.798982    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"]
Sep 17 22:07:01.802888 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.802871    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:01.805160 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.805132    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 22:07:01.805312 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.805189    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-fx5q8\""
Sep 17 22:07:01.805312 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.805226    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5854c-kube-rbac-proxy-sar-config\""
Sep 17 22:07:01.806006 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.805982    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5854c-predictor-serving-cert\""
Sep 17 22:07:01.806160 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.806013    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 22:07:01.810105 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.810083    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"]
Sep 17 22:07:01.936947 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.936910    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svhqg\" (UniqueName: \"kubernetes.io/projected/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-api-access-svhqg\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:01.936947 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.936951    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/055edb3c-2b50-4552-ab32-afbbda2b3748-proxy-tls\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:01.937155 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:01.936990    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.038303 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.038271    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/055edb3c-2b50-4552-ab32-afbbda2b3748-proxy-tls\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.038441 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.038314    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.038441 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.038374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svhqg\" (UniqueName: \"kubernetes.io/projected/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-api-access-svhqg\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.039039 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.039021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.040682 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.040666    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/055edb3c-2b50-4552-ab32-afbbda2b3748-proxy-tls\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.045710 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.045684    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svhqg\" (UniqueName: \"kubernetes.io/projected/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-api-access-svhqg\") pod \"error-404-isvc-5854c-predictor-99d55b49f-hk5lb\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") " pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.114497 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.114434    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:02.382260 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.382219    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"]
Sep 17 22:07:02.384139 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:07:02.384107    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055edb3c_2b50_4552_ab32_afbbda2b3748.slice/crio-50b2692a984ce1657747302de742f4d3d42b103080a7bb5115cc4be6d4585918 WatchSource:0}: Error finding container 50b2692a984ce1657747302de742f4d3d42b103080a7bb5115cc4be6d4585918: Status 404 returned error can't find the container with id 50b2692a984ce1657747302de742f4d3d42b103080a7bb5115cc4be6d4585918
Sep 17 22:07:02.418728 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:02.418699    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" event={"ID":"055edb3c-2b50-4552-ab32-afbbda2b3748","Type":"ContainerStarted","Data":"50b2692a984ce1657747302de742f4d3d42b103080a7bb5115cc4be6d4585918"}
Sep 17 22:07:14.486784 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:14.486742    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" event={"ID":"055edb3c-2b50-4552-ab32-afbbda2b3748","Type":"ContainerStarted","Data":"a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5"}
Sep 17 22:07:16.498923 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:16.498884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" event={"ID":"055edb3c-2b50-4552-ab32-afbbda2b3748","Type":"ContainerStarted","Data":"f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb"}
Sep 17 22:07:16.499297 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:16.499025    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:16.515512 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:16.515466    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podStartSLOduration=1.605528699 podStartE2EDuration="15.51545225s" totalImagesPullingTime="13.909923551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:07:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:07:16.513576299 +0000 UTC m=+567.327951674" watchObservedRunningTime="2026-09-17 22:07:16.51545225 +0000 UTC m=+567.329827627"
Sep 17 22:07:17.503964 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:17.503930    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:17.505262 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:17.505224    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 22:07:18.508045 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:18.508000    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 22:07:23.513995 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:23.513961    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:07:23.514472 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:23.514446    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 22:07:33.514634 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:33.514590    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 22:07:43.514677 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:43.514636    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 22:07:51.097559 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:51.097532    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:07:51.098077 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:51.097708    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:07:53.514932 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:07:53.514891    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 22:08:03.515435 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:03.515403    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:08:21.406402 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.406369    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"]
Sep 17 22:08:21.409532 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.409517    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:21.411753 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.411725    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-5854c-serving-cert\""
Sep 17 22:08:21.411868 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.411820    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-5854c-kube-rbac-proxy-sar-config\""
Sep 17 22:08:21.417520 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.417497    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"]
Sep 17 22:08:21.467455 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.467426    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1225164-c730-4f06-a244-6f7f614af96e-openshift-service-ca-bundle\") pod \"switch-graph-5854c-7cc4867ffc-lv7lz\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") " pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:21.467589 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.467472    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls\") pod \"switch-graph-5854c-7cc4867ffc-lv7lz\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") " pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:21.568076 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.568044    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1225164-c730-4f06-a244-6f7f614af96e-openshift-service-ca-bundle\") pod \"switch-graph-5854c-7cc4867ffc-lv7lz\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") " pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:21.568228 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.568088    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls\") pod \"switch-graph-5854c-7cc4867ffc-lv7lz\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") " pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:21.568228 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:08:21.568189    2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-5854c-serving-cert: secret "switch-graph-5854c-serving-cert" not found
Sep 17 22:08:21.568333 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:08:21.568265    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls podName:d1225164-c730-4f06-a244-6f7f614af96e nodeName:}" failed. No retries permitted until 2026-09-17 22:08:22.068235235 +0000 UTC m=+632.882610591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls") pod "switch-graph-5854c-7cc4867ffc-lv7lz" (UID: "d1225164-c730-4f06-a244-6f7f614af96e") : secret "switch-graph-5854c-serving-cert" not found
Sep 17 22:08:21.568841 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:21.568821    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1225164-c730-4f06-a244-6f7f614af96e-openshift-service-ca-bundle\") pod \"switch-graph-5854c-7cc4867ffc-lv7lz\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") " pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:22.071762 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:22.071726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls\") pod \"switch-graph-5854c-7cc4867ffc-lv7lz\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") " pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:22.074131 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:22.074108    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls\") pod \"switch-graph-5854c-7cc4867ffc-lv7lz\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") " pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:22.320111 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:22.320082    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:22.587121 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:22.587049    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"]
Sep 17 22:08:22.590279 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:08:22.590232    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1225164_c730_4f06_a244_6f7f614af96e.slice/crio-4d7453777c15891efe5061c301080e468989214f5ac37bf2dbb308a757d8954d WatchSource:0}: Error finding container 4d7453777c15891efe5061c301080e468989214f5ac37bf2dbb308a757d8954d: Status 404 returned error can't find the container with id 4d7453777c15891efe5061c301080e468989214f5ac37bf2dbb308a757d8954d
Sep 17 22:08:22.771334 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:22.771302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" event={"ID":"d1225164-c730-4f06-a244-6f7f614af96e","Type":"ContainerStarted","Data":"4d7453777c15891efe5061c301080e468989214f5ac37bf2dbb308a757d8954d"}
Sep 17 22:08:25.786914 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:25.786875    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" event={"ID":"d1225164-c730-4f06-a244-6f7f614af96e","Type":"ContainerStarted","Data":"a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879"}
Sep 17 22:08:25.787308 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:25.786980    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:25.801432 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:25.801387    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podStartSLOduration=2.521685277 podStartE2EDuration="4.80137184s" totalImagesPullingTime="2.279686563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:08:25.800692925 +0000 UTC m=+636.615068301" watchObservedRunningTime="2026-09-17 22:08:25.80137184 +0000 UTC m=+636.615747238"
Sep 17 22:08:31.797353 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:31.797327    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:35.548523 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.548488    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"]
Sep 17 22:08:35.549016 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.548734    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c" containerID="cri-o://a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879" gracePeriod=30
Sep 17 22:08:35.710717 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.710684    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"]
Sep 17 22:08:35.710950 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.710923    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container" containerID="cri-o://a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5" gracePeriod=30
Sep 17 22:08:35.711037 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.710947    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kube-rbac-proxy" containerID="cri-o://f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb" gracePeriod=30
Sep 17 22:08:35.832880 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.832819    2561 generic.go:356] "Generic (PLEG): container finished" podID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerID="f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb" exitCode=2
Sep 17 22:08:35.832880 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.832852    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" event={"ID":"055edb3c-2b50-4552-ab32-afbbda2b3748","Type":"ContainerDied","Data":"f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb"}
Sep 17 22:08:35.869685 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.869659    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"]
Sep 17 22:08:35.872851 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.872834    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:35.874960 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.874934    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8341c-predictor-serving-cert\""
Sep 17 22:08:35.875072 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.874976    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8341c-kube-rbac-proxy-sar-config\""
Sep 17 22:08:35.881864 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.881837    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"]
Sep 17 22:08:35.981891 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.981865    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/872f2a38-fc24-4c38-ae22-922602368804-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:35.982049 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.981898    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf58x\" (UniqueName: \"kubernetes.io/projected/872f2a38-fc24-4c38-ae22-922602368804-kube-api-access-rf58x\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:35.982049 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:35.981930    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/872f2a38-fc24-4c38-ae22-922602368804-proxy-tls\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.083280 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.083185    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/872f2a38-fc24-4c38-ae22-922602368804-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.083280 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.083224    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf58x\" (UniqueName: \"kubernetes.io/projected/872f2a38-fc24-4c38-ae22-922602368804-kube-api-access-rf58x\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.083494 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.083468    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/872f2a38-fc24-4c38-ae22-922602368804-proxy-tls\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.083844 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.083820    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/872f2a38-fc24-4c38-ae22-922602368804-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.085786 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.085767    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/872f2a38-fc24-4c38-ae22-922602368804-proxy-tls\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.090985 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.090951    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf58x\" (UniqueName: \"kubernetes.io/projected/872f2a38-fc24-4c38-ae22-922602368804-kube-api-access-rf58x\") pod \"error-404-isvc-8341c-predictor-b849f7567-l7zl7\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") " pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.183607 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.183580    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.452558 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.452532    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"]
Sep 17 22:08:36.454497 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:08:36.454474    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872f2a38_fc24_4c38_ae22_922602368804.slice/crio-a401d0a1ab22967303c7a23fbaa20c5c523a66ced27e75abe29959aa5aee3785 WatchSource:0}: Error finding container a401d0a1ab22967303c7a23fbaa20c5c523a66ced27e75abe29959aa5aee3785: Status 404 returned error can't find the container with id a401d0a1ab22967303c7a23fbaa20c5c523a66ced27e75abe29959aa5aee3785
Sep 17 22:08:36.796581 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.796499    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:08:36.838463 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.838431    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" event={"ID":"872f2a38-fc24-4c38-ae22-922602368804","Type":"ContainerStarted","Data":"0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0"}
Sep 17 22:08:36.838620 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.838494    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" event={"ID":"872f2a38-fc24-4c38-ae22-922602368804","Type":"ContainerStarted","Data":"8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320"}
Sep 17 22:08:36.838620 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.838521    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.838620 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.838548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" event={"ID":"872f2a38-fc24-4c38-ae22-922602368804","Type":"ContainerStarted","Data":"a401d0a1ab22967303c7a23fbaa20c5c523a66ced27e75abe29959aa5aee3785"}
Sep 17 22:08:36.838782 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.838652    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:36.840079 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.840055    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 22:08:36.854030 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:36.853983    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podStartSLOduration=1.853965485 podStartE2EDuration="1.853965485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:08:36.852332552 +0000 UTC m=+647.666707930" watchObservedRunningTime="2026-09-17 22:08:36.853965485 +0000 UTC m=+647.668340864"
Sep 17 22:08:37.842517 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:37.842482    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 22:08:38.509170 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.509130    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Sep 17 22:08:38.658171 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.658127    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:08:38.702281 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.702255    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-rbac-proxy-sar-config\") pod \"055edb3c-2b50-4552-ab32-afbbda2b3748\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") "
Sep 17 22:08:38.702424 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.702294    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/055edb3c-2b50-4552-ab32-afbbda2b3748-proxy-tls\") pod \"055edb3c-2b50-4552-ab32-afbbda2b3748\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") "
Sep 17 22:08:38.702424 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.702404    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svhqg\" (UniqueName: \"kubernetes.io/projected/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-api-access-svhqg\") pod \"055edb3c-2b50-4552-ab32-afbbda2b3748\" (UID: \"055edb3c-2b50-4552-ab32-afbbda2b3748\") "
Sep 17 22:08:38.702644 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.702622    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "055edb3c-2b50-4552-ab32-afbbda2b3748" (UID: "055edb3c-2b50-4552-ab32-afbbda2b3748"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:08:38.704426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.704405    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-api-access-svhqg" (OuterVolumeSpecName: "kube-api-access-svhqg") pod "055edb3c-2b50-4552-ab32-afbbda2b3748" (UID: "055edb3c-2b50-4552-ab32-afbbda2b3748"). InnerVolumeSpecName "kube-api-access-svhqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:08:38.704524 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.704410    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/055edb3c-2b50-4552-ab32-afbbda2b3748-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "055edb3c-2b50-4552-ab32-afbbda2b3748" (UID: "055edb3c-2b50-4552-ab32-afbbda2b3748"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:08:38.802953 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.802886    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svhqg\" (UniqueName: \"kubernetes.io/projected/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-api-access-svhqg\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:08:38.802953 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.802910    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/055edb3c-2b50-4552-ab32-afbbda2b3748-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:08:38.802953 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.802923    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/055edb3c-2b50-4552-ab32-afbbda2b3748-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:08:38.848065 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.848042    2561 generic.go:356] "Generic (PLEG): container finished" podID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerID="a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5" exitCode=0
Sep 17 22:08:38.848404 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.848094    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" event={"ID":"055edb3c-2b50-4552-ab32-afbbda2b3748","Type":"ContainerDied","Data":"a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5"}
Sep 17 22:08:38.848404 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.848136    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb" event={"ID":"055edb3c-2b50-4552-ab32-afbbda2b3748","Type":"ContainerDied","Data":"50b2692a984ce1657747302de742f4d3d42b103080a7bb5115cc4be6d4585918"}
Sep 17 22:08:38.848404 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.848163    2561 scope.go:117] "RemoveContainer" containerID="f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb"
Sep 17 22:08:38.848404 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.848094    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"
Sep 17 22:08:38.856997 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.856981    2561 scope.go:117] "RemoveContainer" containerID="a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5"
Sep 17 22:08:38.864181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.864163    2561 scope.go:117] "RemoveContainer" containerID="f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb"
Sep 17 22:08:38.864454 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:08:38.864436    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb\": container with ID starting with f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb not found: ID does not exist" containerID="f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb"
Sep 17 22:08:38.864518 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.864462    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb"} err="failed to get container status \"f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb\": rpc error: code = NotFound desc = could not find container \"f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb\": container with ID starting with f40b639cbb552ff70ce4810a13c3628b21b322161b18497c4bd3deed6237bcdb not found: ID does not exist"
Sep 17 22:08:38.864518 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.864478    2561 scope.go:117] "RemoveContainer" containerID="a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5"
Sep 17 22:08:38.864745 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:08:38.864728    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5\": container with ID starting with a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5 not found: ID does not exist" containerID="a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5"
Sep 17 22:08:38.864786 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.864751    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5"} err="failed to get container status \"a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5\": rpc error: code = NotFound desc = could not find container \"a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5\": container with ID starting with a6e32965efac961d4001125c7e01c9243b796dca78ff48c18a8fcc095fbb28c5 not found: ID does not exist"
Sep 17 22:08:38.868638 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.868620    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"]
Sep 17 22:08:38.871853 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:38.871833    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5854c-predictor-99d55b49f-hk5lb"]
Sep 17 22:08:39.136444 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:39.136362    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" path="/var/lib/kubelet/pods/055edb3c-2b50-4552-ab32-afbbda2b3748/volumes"
Sep 17 22:08:41.796370 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:41.796325    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:08:42.847409 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:42.847381    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:08:42.847848 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:42.847823    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 22:08:46.796119 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:46.796080    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:08:46.796500 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:46.796196    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:08:51.796369 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:51.796329    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:08:52.848738 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:52.848703    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 22:08:56.796025 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:08:56.795981    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:09:01.795848 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:01.795811    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:09:02.848828 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:02.848791    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 22:09:05.688845 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.688822    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:09:05.814639 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.814541    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls\") pod \"d1225164-c730-4f06-a244-6f7f614af96e\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") "
Sep 17 22:09:05.814804 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.814656    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1225164-c730-4f06-a244-6f7f614af96e-openshift-service-ca-bundle\") pod \"d1225164-c730-4f06-a244-6f7f614af96e\" (UID: \"d1225164-c730-4f06-a244-6f7f614af96e\") "
Sep 17 22:09:05.815025 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.814997    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1225164-c730-4f06-a244-6f7f614af96e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d1225164-c730-4f06-a244-6f7f614af96e" (UID: "d1225164-c730-4f06-a244-6f7f614af96e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:09:05.816537 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.816512    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d1225164-c730-4f06-a244-6f7f614af96e" (UID: "d1225164-c730-4f06-a244-6f7f614af96e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:09:05.915686 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.915655    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1225164-c730-4f06-a244-6f7f614af96e-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.915686 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.915683    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d1225164-c730-4f06-a244-6f7f614af96e-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:09:05.963937 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.963908    2561 generic.go:356] "Generic (PLEG): container finished" podID="d1225164-c730-4f06-a244-6f7f614af96e" containerID="a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879" exitCode=137
Sep 17 22:09:05.964077 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.963950    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"
Sep 17 22:09:05.964077 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.963997    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" event={"ID":"d1225164-c730-4f06-a244-6f7f614af96e","Type":"ContainerDied","Data":"a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879"}
Sep 17 22:09:05.964077 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.964049    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz" event={"ID":"d1225164-c730-4f06-a244-6f7f614af96e","Type":"ContainerDied","Data":"4d7453777c15891efe5061c301080e468989214f5ac37bf2dbb308a757d8954d"}
Sep 17 22:09:05.964077 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.964076    2561 scope.go:117] "RemoveContainer" containerID="a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879"
Sep 17 22:09:05.972984 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.972970    2561 scope.go:117] "RemoveContainer" containerID="a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879"
Sep 17 22:09:05.973224 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:09:05.973207    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879\": container with ID starting with a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879 not found: ID does not exist" containerID="a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879"
Sep 17 22:09:05.973300 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.973233    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879"} err="failed to get container status \"a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879\": rpc error: code = NotFound desc = could not find container \"a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879\": container with ID starting with a4cc72e414eba096b1da848fc16a1ce12022a97f35dc3c252c238c59e80a8879 not found: ID does not exist"
Sep 17 22:09:05.983312 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.983288    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"]
Sep 17 22:09:05.986805 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:05.986787    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-5854c-7cc4867ffc-lv7lz"]
Sep 17 22:09:07.134471 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:07.134440    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1225164-c730-4f06-a244-6f7f614af96e" path="/var/lib/kubelet/pods/d1225164-c730-4f06-a244-6f7f614af96e/volumes"
Sep 17 22:09:12.847893 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:12.847857    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 22:09:21.385001 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.384971    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"]
Sep 17 22:09:21.385445 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385434    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kube-rbac-proxy"
Sep 17 22:09:21.385512 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385452    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kube-rbac-proxy"
Sep 17 22:09:21.385512 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385467    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container"
Sep 17 22:09:21.385512 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385477    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container"
Sep 17 22:09:21.385512 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385489    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c"
Sep 17 22:09:21.385512 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385497    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c"
Sep 17 22:09:21.385746 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385588    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1225164-c730-4f06-a244-6f7f614af96e" containerName="switch-graph-5854c"
Sep 17 22:09:21.385746 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385601    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kube-rbac-proxy"
Sep 17 22:09:21.385746 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.385612    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="055edb3c-2b50-4552-ab32-afbbda2b3748" containerName="kserve-container"
Sep 17 22:09:21.388773 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.388753    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.390908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.390878    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Sep 17 22:09:21.391002 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.390946    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Sep 17 22:09:21.395661 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.395300    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"]
Sep 17 22:09:21.537713 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.537682    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/41dc5aaf-7e67-4312-8166-694d0034b547-proxy-tls\") pod \"model-chainer-765586574c-cvmzc\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") " pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.537851 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.537723    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dc5aaf-7e67-4312-8166-694d0034b547-openshift-service-ca-bundle\") pod \"model-chainer-765586574c-cvmzc\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") " pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.639146 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.639078    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/41dc5aaf-7e67-4312-8166-694d0034b547-proxy-tls\") pod \"model-chainer-765586574c-cvmzc\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") " pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.639146 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.639114    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dc5aaf-7e67-4312-8166-694d0034b547-openshift-service-ca-bundle\") pod \"model-chainer-765586574c-cvmzc\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") " pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.639758 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.639739    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dc5aaf-7e67-4312-8166-694d0034b547-openshift-service-ca-bundle\") pod \"model-chainer-765586574c-cvmzc\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") " pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.641450 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.641426    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/41dc5aaf-7e67-4312-8166-694d0034b547-proxy-tls\") pod \"model-chainer-765586574c-cvmzc\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") " pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.699426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.699405    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:21.965223 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:21.965198    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"]
Sep 17 22:09:21.967428 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:09:21.967403    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41dc5aaf_7e67_4312_8166_694d0034b547.slice/crio-3e2358e22273033bdccf4f9f36fbba6bf3dab16b04c5c23869132e8f3eaf8a65 WatchSource:0}: Error finding container 3e2358e22273033bdccf4f9f36fbba6bf3dab16b04c5c23869132e8f3eaf8a65: Status 404 returned error can't find the container with id 3e2358e22273033bdccf4f9f36fbba6bf3dab16b04c5c23869132e8f3eaf8a65
Sep 17 22:09:22.032296 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:22.032271    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" event={"ID":"41dc5aaf-7e67-4312-8166-694d0034b547","Type":"ContainerStarted","Data":"9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33"}
Sep 17 22:09:22.032389 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:22.032318    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" event={"ID":"41dc5aaf-7e67-4312-8166-694d0034b547","Type":"ContainerStarted","Data":"3e2358e22273033bdccf4f9f36fbba6bf3dab16b04c5c23869132e8f3eaf8a65"}
Sep 17 22:09:22.032389 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:22.032352    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:22.047485 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:22.047442    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podStartSLOduration=1.047427532 podStartE2EDuration="1.047427532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:09:22.045900698 +0000 UTC m=+692.860276074" watchObservedRunningTime="2026-09-17 22:09:22.047427532 +0000 UTC m=+692.861802958"
Sep 17 22:09:22.849103 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:22.849073    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:09:28.042675 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:28.042641    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:31.918739 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:31.918705    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"]
Sep 17 22:09:31.919202 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:31.918873    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer" containerID="cri-o://9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33" gracePeriod=30
Sep 17 22:09:32.294202 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.294126    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"]
Sep 17 22:09:32.298014 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.297997    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.300239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.300216    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a4672-predictor-serving-cert\""
Sep 17 22:09:32.300387 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.300279    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a4672-kube-rbac-proxy-sar-config\""
Sep 17 22:09:32.305869 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.305842    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"]
Sep 17 22:09:32.321341 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.321304    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c23e2670-b897-425f-b746-f37006aab637-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.321464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.321445    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvqvf\" (UniqueName: \"kubernetes.io/projected/c23e2670-b897-425f-b746-f37006aab637-kube-api-access-qvqvf\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.321544 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.321524    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.422462 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.422437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvqvf\" (UniqueName: \"kubernetes.io/projected/c23e2670-b897-425f-b746-f37006aab637-kube-api-access-qvqvf\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.422609 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.422488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.422609 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.422525    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c23e2670-b897-425f-b746-f37006aab637-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.422722 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:09:32.422649    2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a4672-predictor-serving-cert: secret "error-404-isvc-a4672-predictor-serving-cert" not found
Sep 17 22:09:32.422777 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:09:32.422727    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls podName:c23e2670-b897-425f-b746-f37006aab637 nodeName:}" failed. No retries permitted until 2026-09-17 22:09:32.922704549 +0000 UTC m=+703.737079910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls") pod "error-404-isvc-a4672-predictor-7f59856fb5-d29mp" (UID: "c23e2670-b897-425f-b746-f37006aab637") : secret "error-404-isvc-a4672-predictor-serving-cert" not found
Sep 17 22:09:32.423171 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.423153    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c23e2670-b897-425f-b746-f37006aab637-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.432362 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.432336    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvqvf\" (UniqueName: \"kubernetes.io/projected/c23e2670-b897-425f-b746-f37006aab637-kube-api-access-qvqvf\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.926592 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.926560    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:32.928996 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:32.928969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls\") pod \"error-404-isvc-a4672-predictor-7f59856fb5-d29mp\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") " pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:33.040581 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:33.040543    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:09:33.209357 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:33.209271    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:33.478406 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:33.478379    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"]
Sep 17 22:09:33.480259 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:09:33.480214    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23e2670_b897_425f_b746_f37006aab637.slice/crio-e8a524975ba1fd89c85f54a4f55c229166bbfe5dff1ab1ac2ffeed80cda828b4 WatchSource:0}: Error finding container e8a524975ba1fd89c85f54a4f55c229166bbfe5dff1ab1ac2ffeed80cda828b4: Status 404 returned error can't find the container with id e8a524975ba1fd89c85f54a4f55c229166bbfe5dff1ab1ac2ffeed80cda828b4
Sep 17 22:09:34.088904 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:34.088865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" event={"ID":"c23e2670-b897-425f-b746-f37006aab637","Type":"ContainerStarted","Data":"cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df"}
Sep 17 22:09:34.089287 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:34.088926    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" event={"ID":"c23e2670-b897-425f-b746-f37006aab637","Type":"ContainerStarted","Data":"ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211"}
Sep 17 22:09:34.089287 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:34.088951    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" event={"ID":"c23e2670-b897-425f-b746-f37006aab637","Type":"ContainerStarted","Data":"e8a524975ba1fd89c85f54a4f55c229166bbfe5dff1ab1ac2ffeed80cda828b4"}
Sep 17 22:09:34.089287 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:34.088977    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:34.105553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:34.105514    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podStartSLOduration=2.105497253 podStartE2EDuration="2.105497253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:09:34.104408024 +0000 UTC m=+704.918783401" watchObservedRunningTime="2026-09-17 22:09:34.105497253 +0000 UTC m=+704.919872629"
Sep 17 22:09:35.093409 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.093379    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:35.094863 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.094833    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 22:09:35.772183 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.772155    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"]
Sep 17 22:09:35.775477 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.775461    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:35.777457 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.777435    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8341c-kube-rbac-proxy-sar-config\""
Sep 17 22:09:35.777561 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.777440    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8341c-serving-cert\""
Sep 17 22:09:35.784441 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.784421    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"]
Sep 17 22:09:35.850132 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.850106    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9313f174-5178-4200-a81a-630ac3c7e093-openshift-service-ca-bundle\") pod \"switch-graph-8341c-567d856458-h2xgg\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") " pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:35.850279 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.850170    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9313f174-5178-4200-a81a-630ac3c7e093-proxy-tls\") pod \"switch-graph-8341c-567d856458-h2xgg\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") " pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:35.950535 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.950506    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9313f174-5178-4200-a81a-630ac3c7e093-proxy-tls\") pod \"switch-graph-8341c-567d856458-h2xgg\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") " pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:35.950660 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.950567    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9313f174-5178-4200-a81a-630ac3c7e093-openshift-service-ca-bundle\") pod \"switch-graph-8341c-567d856458-h2xgg\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") " pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:35.951209 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.951189    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9313f174-5178-4200-a81a-630ac3c7e093-openshift-service-ca-bundle\") pod \"switch-graph-8341c-567d856458-h2xgg\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") " pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:35.952829 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:35.952805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9313f174-5178-4200-a81a-630ac3c7e093-proxy-tls\") pod \"switch-graph-8341c-567d856458-h2xgg\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") " pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:36.087095 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:36.087024    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:36.100911 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:36.100879    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 22:09:36.358829 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:36.358667    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"]
Sep 17 22:09:36.361387 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:09:36.361355    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9313f174_5178_4200_a81a_630ac3c7e093.slice/crio-af1219f1fdffbb17291e2aa230271d547d00ff4674505d061e30bfc5181eb264 WatchSource:0}: Error finding container af1219f1fdffbb17291e2aa230271d547d00ff4674505d061e30bfc5181eb264: Status 404 returned error can't find the container with id af1219f1fdffbb17291e2aa230271d547d00ff4674505d061e30bfc5181eb264
Sep 17 22:09:37.105867 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:37.105826    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" event={"ID":"9313f174-5178-4200-a81a-630ac3c7e093","Type":"ContainerStarted","Data":"13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7"}
Sep 17 22:09:37.106281 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:37.105918    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" event={"ID":"9313f174-5178-4200-a81a-630ac3c7e093","Type":"ContainerStarted","Data":"af1219f1fdffbb17291e2aa230271d547d00ff4674505d061e30bfc5181eb264"}
Sep 17 22:09:37.106281 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:37.105975    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:37.120947 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:37.120904    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podStartSLOduration=2.120891524 podStartE2EDuration="2.120891524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:09:37.119619087 +0000 UTC m=+707.933994464" watchObservedRunningTime="2026-09-17 22:09:37.120891524 +0000 UTC m=+707.935266923"
Sep 17 22:09:38.040972 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:38.040935    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:09:41.106482 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:41.106454    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:09:41.106871 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:41.106818    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 22:09:43.041239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:43.041201    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:09:43.041614 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:43.041323    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:09:43.115849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:43.115822    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:09:48.041306 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:48.041262    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:09:51.107120 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:51.107084    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 22:09:53.041202 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:53.041163    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:09:58.040488 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:09:58.040450    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:10:01.107611 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:01.107571    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 22:10:02.060620 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.060595    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:10:02.165239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.165213    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dc5aaf-7e67-4312-8166-694d0034b547-openshift-service-ca-bundle\") pod \"41dc5aaf-7e67-4312-8166-694d0034b547\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") "
Sep 17 22:10:02.165616 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.165266    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/41dc5aaf-7e67-4312-8166-694d0034b547-proxy-tls\") pod \"41dc5aaf-7e67-4312-8166-694d0034b547\" (UID: \"41dc5aaf-7e67-4312-8166-694d0034b547\") "
Sep 17 22:10:02.165616 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.165551    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dc5aaf-7e67-4312-8166-694d0034b547-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "41dc5aaf-7e67-4312-8166-694d0034b547" (UID: "41dc5aaf-7e67-4312-8166-694d0034b547"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:10:02.167231 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.167210    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dc5aaf-7e67-4312-8166-694d0034b547-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "41dc5aaf-7e67-4312-8166-694d0034b547" (UID: "41dc5aaf-7e67-4312-8166-694d0034b547"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:10:02.210289 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.210207    2561 generic.go:356] "Generic (PLEG): container finished" podID="41dc5aaf-7e67-4312-8166-694d0034b547" containerID="9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33" exitCode=137
Sep 17 22:10:02.210289 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.210267    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"
Sep 17 22:10:02.210448 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.210279    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" event={"ID":"41dc5aaf-7e67-4312-8166-694d0034b547","Type":"ContainerDied","Data":"9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33"}
Sep 17 22:10:02.210448 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.210335    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-765586574c-cvmzc" event={"ID":"41dc5aaf-7e67-4312-8166-694d0034b547","Type":"ContainerDied","Data":"3e2358e22273033bdccf4f9f36fbba6bf3dab16b04c5c23869132e8f3eaf8a65"}
Sep 17 22:10:02.210448 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.210359    2561 scope.go:117] "RemoveContainer" containerID="9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33"
Sep 17 22:10:02.219160 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.219145    2561 scope.go:117] "RemoveContainer" containerID="9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33"
Sep 17 22:10:02.219440 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:10:02.219420    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33\": container with ID starting with 9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33 not found: ID does not exist" containerID="9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33"
Sep 17 22:10:02.219492 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.219448    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33"} err="failed to get container status \"9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33\": rpc error: code = NotFound desc = could not find container \"9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33\": container with ID starting with 9a4c0b346f8eaf9f4fd4192200214ca8d1aeab2bf43a774a05665741912f6c33 not found: ID does not exist"
Sep 17 22:10:02.230524 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.230500    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"]
Sep 17 22:10:02.234107 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.234090    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-765586574c-cvmzc"]
Sep 17 22:10:02.266521 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.266490    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dc5aaf-7e67-4312-8166-694d0034b547-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:10:02.266521 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:02.266522    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/41dc5aaf-7e67-4312-8166-694d0034b547-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:10:03.132773 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:03.132744    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" path="/var/lib/kubelet/pods/41dc5aaf-7e67-4312-8166-694d0034b547/volumes"
Sep 17 22:10:11.107444 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:11.107408    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 22:10:21.108397 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:21.108367    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:10:32.127872 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.127837    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"]
Sep 17 22:10:32.128390 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.128375    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer"
Sep 17 22:10:32.128436 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.128393    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer"
Sep 17 22:10:32.128485 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.128474    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dc5aaf-7e67-4312-8166-694d0034b547" containerName="model-chainer"
Sep 17 22:10:32.131598 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.131581    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:32.133688 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.133662    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a4672-serving-cert\""
Sep 17 22:10:32.133688 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.133680    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a4672-kube-rbac-proxy-sar-config\""
Sep 17 22:10:32.136889 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.136865    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"]
Sep 17 22:10:32.185587 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.185560    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls\") pod \"sequence-graph-a4672-66f4cddcbb-qrgh9\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") " pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:32.185711 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.185614    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b3180f-05fc-43d2-95bc-46cae51026eb-openshift-service-ca-bundle\") pod \"sequence-graph-a4672-66f4cddcbb-qrgh9\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") " pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:32.286823 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.286798    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls\") pod \"sequence-graph-a4672-66f4cddcbb-qrgh9\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") " pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:32.286930 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.286840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b3180f-05fc-43d2-95bc-46cae51026eb-openshift-service-ca-bundle\") pod \"sequence-graph-a4672-66f4cddcbb-qrgh9\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") " pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:32.286977 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:10:32.286936    2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-a4672-serving-cert: secret "sequence-graph-a4672-serving-cert" not found
Sep 17 22:10:32.287013 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:10:32.286996    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls podName:93b3180f-05fc-43d2-95bc-46cae51026eb nodeName:}" failed. No retries permitted until 2026-09-17 22:10:32.786979259 +0000 UTC m=+763.601354617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls") pod "sequence-graph-a4672-66f4cddcbb-qrgh9" (UID: "93b3180f-05fc-43d2-95bc-46cae51026eb") : secret "sequence-graph-a4672-serving-cert" not found
Sep 17 22:10:32.287407 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.287391    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b3180f-05fc-43d2-95bc-46cae51026eb-openshift-service-ca-bundle\") pod \"sequence-graph-a4672-66f4cddcbb-qrgh9\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") " pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:32.790987 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.790952    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls\") pod \"sequence-graph-a4672-66f4cddcbb-qrgh9\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") " pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:32.793351 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:32.793320    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls\") pod \"sequence-graph-a4672-66f4cddcbb-qrgh9\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") " pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:33.042426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:33.042348    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:33.308909 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:33.308885    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"]
Sep 17 22:10:33.311650 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:10:33.311622    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b3180f_05fc_43d2_95bc_46cae51026eb.slice/crio-16e884254a97a37d1d820c014e5d4628720ee56a6d01dd8253a4b84e82592137 WatchSource:0}: Error finding container 16e884254a97a37d1d820c014e5d4628720ee56a6d01dd8253a4b84e82592137: Status 404 returned error can't find the container with id 16e884254a97a37d1d820c014e5d4628720ee56a6d01dd8253a4b84e82592137
Sep 17 22:10:33.338089 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:33.338057    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" event={"ID":"93b3180f-05fc-43d2-95bc-46cae51026eb","Type":"ContainerStarted","Data":"16e884254a97a37d1d820c014e5d4628720ee56a6d01dd8253a4b84e82592137"}
Sep 17 22:10:34.345073 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:34.345034    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" event={"ID":"93b3180f-05fc-43d2-95bc-46cae51026eb","Type":"ContainerStarted","Data":"a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2"}
Sep 17 22:10:34.345514 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:34.345097    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:10:34.360408 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:34.360365    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podStartSLOduration=2.360351853 podStartE2EDuration="2.360351853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:10:34.358917225 +0000 UTC m=+765.173292604" watchObservedRunningTime="2026-09-17 22:10:34.360351853 +0000 UTC m=+765.174727227"
Sep 17 22:10:40.355043 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:10:40.355011    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:12:51.123113 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:12:51.123044    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:12:51.124517 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:12:51.124498    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:17:50.450164 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.450131    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"]
Sep 17 22:17:50.450764 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.450396    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c" containerID="cri-o://13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7" gracePeriod=30
Sep 17 22:17:50.620084 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.620053    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"]
Sep 17 22:17:50.620351 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.620297    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" containerID="cri-o://8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320" gracePeriod=30
Sep 17 22:17:50.620496 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.620323    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kube-rbac-proxy" containerID="cri-o://0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0" gracePeriod=30
Sep 17 22:17:50.719415 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.719341    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"]
Sep 17 22:17:50.723026 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.723004    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:50.725243 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.725216    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ab288-kube-rbac-proxy-sar-config\""
Sep 17 22:17:50.725508 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.725493    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ab288-predictor-serving-cert\""
Sep 17 22:17:50.730187 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.730163    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"]
Sep 17 22:17:50.904195 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.904163    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50d0de5a-be87-4994-9c37-42800d1c7f61-proxy-tls\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:50.904385 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.904206    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:50.904385 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:50.904273    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qv4b\" (UniqueName: \"kubernetes.io/projected/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-api-access-7qv4b\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.005344 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.005243    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qv4b\" (UniqueName: \"kubernetes.io/projected/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-api-access-7qv4b\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.005344 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.005330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50d0de5a-be87-4994-9c37-42800d1c7f61-proxy-tls\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.005552 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.005366    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.006026 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.006002    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.007791 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.007769    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50d0de5a-be87-4994-9c37-42800d1c7f61-proxy-tls\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.012392 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.012375    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qv4b\" (UniqueName: \"kubernetes.io/projected/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-api-access-7qv4b\") pod \"error-404-isvc-ab288-predictor-c9f568f7f-nbvxq\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") " pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.036521 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.036500    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:51.149359 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.149335    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:17:51.151982 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.151959    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:17:51.205099 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.205069    2561 generic.go:356] "Generic (PLEG): container finished" podID="872f2a38-fc24-4c38-ae22-922602368804" containerID="0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0" exitCode=2
Sep 17 22:17:51.205265 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.205109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" event={"ID":"872f2a38-fc24-4c38-ae22-922602368804","Type":"ContainerDied","Data":"0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0"}
Sep 17 22:17:51.320954 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:51.320924    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"]
Sep 17 22:17:51.322689 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:17:51.322662    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d0de5a_be87_4994_9c37_42800d1c7f61.slice/crio-8c207771ad1173973af75e34898ebee11d1fe28d6f3ba2e7b46b946dc7772d13 WatchSource:0}: Error finding container 8c207771ad1173973af75e34898ebee11d1fe28d6f3ba2e7b46b946dc7772d13: Status 404 returned error can't find the container with id 8c207771ad1173973af75e34898ebee11d1fe28d6f3ba2e7b46b946dc7772d13
Sep 17 22:17:52.211929 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.211890    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" event={"ID":"50d0de5a-be87-4994-9c37-42800d1c7f61","Type":"ContainerStarted","Data":"6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a"}
Sep 17 22:17:52.212374 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.211957    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" event={"ID":"50d0de5a-be87-4994-9c37-42800d1c7f61","Type":"ContainerStarted","Data":"3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09"}
Sep 17 22:17:52.212374 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.211988    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" event={"ID":"50d0de5a-be87-4994-9c37-42800d1c7f61","Type":"ContainerStarted","Data":"8c207771ad1173973af75e34898ebee11d1fe28d6f3ba2e7b46b946dc7772d13"}
Sep 17 22:17:52.212374 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.212021    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:52.212374 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.212049    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:52.213590 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.213567    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 22:17:52.228504 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.228460    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podStartSLOduration=2.228445141 podStartE2EDuration="2.228445141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:17:52.226530552 +0000 UTC m=+1203.040905929" watchObservedRunningTime="2026-09-17 22:17:52.228445141 +0000 UTC m=+1203.042820518"
Sep 17 22:17:52.843157 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.843122    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": dial tcp 10.133.0.32:8643: connect: connection refused"
Sep 17 22:17:52.848037 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:52.848013    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 22:17:53.114237 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.114160    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:17:53.216790 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.216750    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 22:17:53.554259 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.554221    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:17:53.726044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.726012    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/872f2a38-fc24-4c38-ae22-922602368804-kube-rbac-proxy-sar-config\") pod \"872f2a38-fc24-4c38-ae22-922602368804\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") "
Sep 17 22:17:53.726231 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.726054    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/872f2a38-fc24-4c38-ae22-922602368804-proxy-tls\") pod \"872f2a38-fc24-4c38-ae22-922602368804\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") "
Sep 17 22:17:53.726231 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.726103    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf58x\" (UniqueName: \"kubernetes.io/projected/872f2a38-fc24-4c38-ae22-922602368804-kube-api-access-rf58x\") pod \"872f2a38-fc24-4c38-ae22-922602368804\" (UID: \"872f2a38-fc24-4c38-ae22-922602368804\") "
Sep 17 22:17:53.726464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.726430    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872f2a38-fc24-4c38-ae22-922602368804-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "872f2a38-fc24-4c38-ae22-922602368804" (UID: "872f2a38-fc24-4c38-ae22-922602368804"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:17:53.728197 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.728170    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872f2a38-fc24-4c38-ae22-922602368804-kube-api-access-rf58x" (OuterVolumeSpecName: "kube-api-access-rf58x") pod "872f2a38-fc24-4c38-ae22-922602368804" (UID: "872f2a38-fc24-4c38-ae22-922602368804"). InnerVolumeSpecName "kube-api-access-rf58x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:17:53.728300 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.728219    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872f2a38-fc24-4c38-ae22-922602368804-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "872f2a38-fc24-4c38-ae22-922602368804" (UID: "872f2a38-fc24-4c38-ae22-922602368804"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:17:53.826776 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.826733    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/872f2a38-fc24-4c38-ae22-922602368804-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:17:53.826776 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.826769    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/872f2a38-fc24-4c38-ae22-922602368804-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:17:53.826776 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:53.826779    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf58x\" (UniqueName: \"kubernetes.io/projected/872f2a38-fc24-4c38-ae22-922602368804-kube-api-access-rf58x\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:17:54.222823 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.222792    2561 generic.go:356] "Generic (PLEG): container finished" podID="872f2a38-fc24-4c38-ae22-922602368804" containerID="8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320" exitCode=0
Sep 17 22:17:54.223204 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.222843    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"
Sep 17 22:17:54.223204 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.222851    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" event={"ID":"872f2a38-fc24-4c38-ae22-922602368804","Type":"ContainerDied","Data":"8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320"}
Sep 17 22:17:54.223204 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.222907    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7" event={"ID":"872f2a38-fc24-4c38-ae22-922602368804","Type":"ContainerDied","Data":"a401d0a1ab22967303c7a23fbaa20c5c523a66ced27e75abe29959aa5aee3785"}
Sep 17 22:17:54.223204 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.222936    2561 scope.go:117] "RemoveContainer" containerID="0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0"
Sep 17 22:17:54.231874 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.231859    2561 scope.go:117] "RemoveContainer" containerID="8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320"
Sep 17 22:17:54.238994 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.238977    2561 scope.go:117] "RemoveContainer" containerID="0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0"
Sep 17 22:17:54.239268 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:17:54.239236    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0\": container with ID starting with 0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0 not found: ID does not exist" containerID="0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0"
Sep 17 22:17:54.239320 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.239279    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0"} err="failed to get container status \"0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0\": rpc error: code = NotFound desc = could not find container \"0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0\": container with ID starting with 0798c5c629322cdf25e9025fe58a9b3821c37344c5c62add4673d9d2f178f1e0 not found: ID does not exist"
Sep 17 22:17:54.239320 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.239296    2561 scope.go:117] "RemoveContainer" containerID="8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320"
Sep 17 22:17:54.239537 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:17:54.239520    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320\": container with ID starting with 8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320 not found: ID does not exist" containerID="8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320"
Sep 17 22:17:54.239574 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.239543    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320"} err="failed to get container status \"8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320\": rpc error: code = NotFound desc = could not find container \"8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320\": container with ID starting with 8a5ef4c648d0baa13a280bd07a8d80e98a9e26bd49ab2236f0f9a434035ac320 not found: ID does not exist"
Sep 17 22:17:54.242692 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.242671    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"]
Sep 17 22:17:54.246125 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:54.246103    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8341c-predictor-b849f7567-l7zl7"]
Sep 17 22:17:55.132872 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:55.132842    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="872f2a38-fc24-4c38-ae22-922602368804" path="/var/lib/kubelet/pods/872f2a38-fc24-4c38-ae22-922602368804/volumes"
Sep 17 22:17:58.114181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:58.114143    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:17:58.221132 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:58.221101    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:17:58.221579 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:17:58.221553    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 22:18:03.114378 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:03.114338    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:18:03.114733 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:03.114468    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:18:08.114630 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:08.114595    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:18:08.221518 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:08.221477    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 22:18:13.114854 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:13.114818    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:18:18.114696 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:18.114659    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:18:18.222277 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:18.222222    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 22:18:20.585805 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:20.585780    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:18:20.632850 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:20.632825    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9313f174-5178-4200-a81a-630ac3c7e093-openshift-service-ca-bundle\") pod \"9313f174-5178-4200-a81a-630ac3c7e093\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") "
Sep 17 22:18:20.632973 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:20.632857    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9313f174-5178-4200-a81a-630ac3c7e093-proxy-tls\") pod \"9313f174-5178-4200-a81a-630ac3c7e093\" (UID: \"9313f174-5178-4200-a81a-630ac3c7e093\") "
Sep 17 22:18:20.633230 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:20.633200    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9313f174-5178-4200-a81a-630ac3c7e093-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9313f174-5178-4200-a81a-630ac3c7e093" (UID: "9313f174-5178-4200-a81a-630ac3c7e093"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:18:20.634872 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:20.634851    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9313f174-5178-4200-a81a-630ac3c7e093-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9313f174-5178-4200-a81a-630ac3c7e093" (UID: "9313f174-5178-4200-a81a-630ac3c7e093"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:18:20.734136 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:20.734074    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9313f174-5178-4200-a81a-630ac3c7e093-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:18:20.734136 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:20.734097    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9313f174-5178-4200-a81a-630ac3c7e093-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:18:21.343709 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.343681    2561 generic.go:356] "Generic (PLEG): container finished" podID="9313f174-5178-4200-a81a-630ac3c7e093" containerID="13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7" exitCode=0
Sep 17 22:18:21.343924 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.343725    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"
Sep 17 22:18:21.343924 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.343733    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" event={"ID":"9313f174-5178-4200-a81a-630ac3c7e093","Type":"ContainerDied","Data":"13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7"}
Sep 17 22:18:21.343924 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.343782    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg" event={"ID":"9313f174-5178-4200-a81a-630ac3c7e093","Type":"ContainerDied","Data":"af1219f1fdffbb17291e2aa230271d547d00ff4674505d061e30bfc5181eb264"}
Sep 17 22:18:21.343924 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.343805    2561 scope.go:117] "RemoveContainer" containerID="13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7"
Sep 17 22:18:21.352597 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.352580    2561 scope.go:117] "RemoveContainer" containerID="13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7"
Sep 17 22:18:21.352891 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:18:21.352873    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7\": container with ID starting with 13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7 not found: ID does not exist" containerID="13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7"
Sep 17 22:18:21.352940 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.352899    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7"} err="failed to get container status \"13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7\": rpc error: code = NotFound desc = could not find container \"13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7\": container with ID starting with 13a187882792d9ab1491b06dec2846442f17216100b2cd61e481959a9abd99c7 not found: ID does not exist"
Sep 17 22:18:21.359699 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.359668    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"]
Sep 17 22:18:21.363147 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:21.363126    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8341c-567d856458-h2xgg"]
Sep 17 22:18:23.132911 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:23.132877    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9313f174-5178-4200-a81a-630ac3c7e093" path="/var/lib/kubelet/pods/9313f174-5178-4200-a81a-630ac3c7e093/volumes"
Sep 17 22:18:28.222208 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:28.222167    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 22:18:38.223044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:38.222972    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:18:46.688977 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.688945    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"]
Sep 17 22:18:46.689421 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.689150    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672" containerID="cri-o://a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2" gracePeriod=30
Sep 17 22:18:46.808498 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.808462    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"]
Sep 17 22:18:46.808756 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.808704    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container" containerID="cri-o://ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211" gracePeriod=30
Sep 17 22:18:46.808890 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.808742    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kube-rbac-proxy" containerID="cri-o://cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df" gracePeriod=30
Sep 17 22:18:46.980316 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980226    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"]
Sep 17 22:18:46.980750 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980728    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kube-rbac-proxy"
Sep 17 22:18:46.980750 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980746    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kube-rbac-proxy"
Sep 17 22:18:46.980925 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980771    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container"
Sep 17 22:18:46.980925 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980780    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container"
Sep 17 22:18:46.980925 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980793    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c"
Sep 17 22:18:46.980925 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980803    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c"
Sep 17 22:18:46.980925 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980875    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9313f174-5178-4200-a81a-630ac3c7e093" containerName="switch-graph-8341c"
Sep 17 22:18:46.980925 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980891    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kserve-container"
Sep 17 22:18:46.980925 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.980901    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="872f2a38-fc24-4c38-ae22-922602368804" containerName="kube-rbac-proxy"
Sep 17 22:18:46.984282 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.984237    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:46.986527 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.986506    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1fd26-predictor-serving-cert\""
Sep 17 22:18:46.986645 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.986523    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1fd26-kube-rbac-proxy-sar-config\""
Sep 17 22:18:46.992855 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:46.992785    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"]
Sep 17 22:18:47.034540 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.034515    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq5mb\" (UniqueName: \"kubernetes.io/projected/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-api-access-cq5mb\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.034683 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.034547    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.034683 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.034575    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.135515 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.135488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.135671 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.135555    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq5mb\" (UniqueName: \"kubernetes.io/projected/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-api-access-cq5mb\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.135671 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.135581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.135767 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:18:47.135691    2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-serving-cert: secret "error-404-isvc-1fd26-predictor-serving-cert" not found
Sep 17 22:18:47.135767 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:18:47.135755    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls podName:4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e nodeName:}" failed. No retries permitted until 2026-09-17 22:18:47.635737703 +0000 UTC m=+1258.450113072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls") pod "error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" (UID: "4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e") : secret "error-404-isvc-1fd26-predictor-serving-cert" not found
Sep 17 22:18:47.136266 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.136225    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.145703 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.145678    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq5mb\" (UniqueName: \"kubernetes.io/projected/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-api-access-cq5mb\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.461483 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.461451    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23e2670-b897-425f-b746-f37006aab637" containerID="cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df" exitCode=2
Sep 17 22:18:47.461619 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.461495    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" event={"ID":"c23e2670-b897-425f-b746-f37006aab637","Type":"ContainerDied","Data":"cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df"}
Sep 17 22:18:47.639789 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.639756    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.642055 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.642036    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls\") pod \"error-404-isvc-1fd26-predictor-64d658b8f5-b29nl\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") " pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:47.897345 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:47.897235    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:48.165901 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.165873    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"]
Sep 17 22:18:48.168022 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:18:48.167995    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e51baa5_18e4_4d6d_9fd8_52d510c3dc5e.slice/crio-140e92f8e4461c4286db5a31748c52a3c3c6dfca215f4d514470661f4105b71a WatchSource:0}: Error finding container 140e92f8e4461c4286db5a31748c52a3c3c6dfca215f4d514470661f4105b71a: Status 404 returned error can't find the container with id 140e92f8e4461c4286db5a31748c52a3c3c6dfca215f4d514470661f4105b71a
Sep 17 22:18:48.467835 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.467754    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" event={"ID":"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e","Type":"ContainerStarted","Data":"296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c"}
Sep 17 22:18:48.467835 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.467806    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" event={"ID":"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e","Type":"ContainerStarted","Data":"7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8"}
Sep 17 22:18:48.467835 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.467827    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" event={"ID":"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e","Type":"ContainerStarted","Data":"140e92f8e4461c4286db5a31748c52a3c3c6dfca215f4d514470661f4105b71a"}
Sep 17 22:18:48.468086 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.467851    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:48.468086 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.467899    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:48.469298 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.469272    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 22:18:48.484518 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:48.484470    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podStartSLOduration=2.484453464 podStartE2EDuration="2.484453464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:18:48.48268192 +0000 UTC m=+1259.297057297" watchObservedRunningTime="2026-09-17 22:18:48.484453464 +0000 UTC m=+1259.298828842"
Sep 17 22:18:49.471930 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:49.471892    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 22:18:50.152695 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.152671    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:18:50.260635 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.260563    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls\") pod \"c23e2670-b897-425f-b746-f37006aab637\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") "
Sep 17 22:18:50.260635 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.260601    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c23e2670-b897-425f-b746-f37006aab637-kube-rbac-proxy-sar-config\") pod \"c23e2670-b897-425f-b746-f37006aab637\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") "
Sep 17 22:18:50.260832 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.260688    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvqvf\" (UniqueName: \"kubernetes.io/projected/c23e2670-b897-425f-b746-f37006aab637-kube-api-access-qvqvf\") pod \"c23e2670-b897-425f-b746-f37006aab637\" (UID: \"c23e2670-b897-425f-b746-f37006aab637\") "
Sep 17 22:18:50.260998 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.260976    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23e2670-b897-425f-b746-f37006aab637-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c23e2670-b897-425f-b746-f37006aab637" (UID: "c23e2670-b897-425f-b746-f37006aab637"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:18:50.262798 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.262770    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c23e2670-b897-425f-b746-f37006aab637" (UID: "c23e2670-b897-425f-b746-f37006aab637"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:18:50.262908 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.262873    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23e2670-b897-425f-b746-f37006aab637-kube-api-access-qvqvf" (OuterVolumeSpecName: "kube-api-access-qvqvf") pod "c23e2670-b897-425f-b746-f37006aab637" (UID: "c23e2670-b897-425f-b746-f37006aab637"). InnerVolumeSpecName "kube-api-access-qvqvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:18:50.353204 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.353173    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:18:50.361690 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.361664    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c23e2670-b897-425f-b746-f37006aab637-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:18:50.361690 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.361689    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c23e2670-b897-425f-b746-f37006aab637-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:18:50.361813 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.361702    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvqvf\" (UniqueName: \"kubernetes.io/projected/c23e2670-b897-425f-b746-f37006aab637-kube-api-access-qvqvf\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:18:50.477405 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.477378    2561 generic.go:356] "Generic (PLEG): container finished" podID="c23e2670-b897-425f-b746-f37006aab637" containerID="ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211" exitCode=0
Sep 17 22:18:50.477766 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.477447    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"
Sep 17 22:18:50.477766 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.477467    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" event={"ID":"c23e2670-b897-425f-b746-f37006aab637","Type":"ContainerDied","Data":"ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211"}
Sep 17 22:18:50.477766 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.477525    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp" event={"ID":"c23e2670-b897-425f-b746-f37006aab637","Type":"ContainerDied","Data":"e8a524975ba1fd89c85f54a4f55c229166bbfe5dff1ab1ac2ffeed80cda828b4"}
Sep 17 22:18:50.477766 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.477555    2561 scope.go:117] "RemoveContainer" containerID="cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df"
Sep 17 22:18:50.486219 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.486204    2561 scope.go:117] "RemoveContainer" containerID="ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211"
Sep 17 22:18:50.493717 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.493700    2561 scope.go:117] "RemoveContainer" containerID="cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df"
Sep 17 22:18:50.493991 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:18:50.493972    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df\": container with ID starting with cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df not found: ID does not exist" containerID="cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df"
Sep 17 22:18:50.494065 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.494003    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df"} err="failed to get container status \"cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df\": rpc error: code = NotFound desc = could not find container \"cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df\": container with ID starting with cd59dac270042fc10c613a43fbe56cca24353a1dddde2480227f327d3dace4df not found: ID does not exist"
Sep 17 22:18:50.494065 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.494025    2561 scope.go:117] "RemoveContainer" containerID="ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211"
Sep 17 22:18:50.494319 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:18:50.494296    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211\": container with ID starting with ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211 not found: ID does not exist" containerID="ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211"
Sep 17 22:18:50.494403 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.494326    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211"} err="failed to get container status \"ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211\": rpc error: code = NotFound desc = could not find container \"ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211\": container with ID starting with ea063b80b7ba759d6c5f704969d7f4c43dff113bab4ed7d465efa50cee20c211 not found: ID does not exist"
Sep 17 22:18:50.499422 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.499401    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"]
Sep 17 22:18:50.503439 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.503418    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4672-predictor-7f59856fb5-d29mp"]
Sep 17 22:18:50.682553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.678348    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"]
Sep 17 22:18:50.682553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.679112    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kube-rbac-proxy"
Sep 17 22:18:50.682553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.679131    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kube-rbac-proxy"
Sep 17 22:18:50.682553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.679156    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container"
Sep 17 22:18:50.682553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.679166    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container"
Sep 17 22:18:50.682553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.679331    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kube-rbac-proxy"
Sep 17 22:18:50.682553 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.679353    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23e2670-b897-425f-b746-f37006aab637" containerName="kserve-container"
Sep 17 22:18:50.685159 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.685056    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:50.687403 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.687378    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-ab288-kube-rbac-proxy-sar-config\""
Sep 17 22:18:50.687403 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.687378    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-ab288-serving-cert\""
Sep 17 22:18:50.688491 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.688465    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"]
Sep 17 22:18:50.766387 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.766350    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b006ed1e-3b9d-4974-b09a-e5025a0a6268-openshift-service-ca-bundle\") pod \"ensemble-graph-ab288-7cb46799f-b5llf\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") " pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:50.766546 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.766401    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls\") pod \"ensemble-graph-ab288-7cb46799f-b5llf\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") " pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:50.866874 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.866846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls\") pod \"ensemble-graph-ab288-7cb46799f-b5llf\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") " pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:50.867028 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.866924    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b006ed1e-3b9d-4974-b09a-e5025a0a6268-openshift-service-ca-bundle\") pod \"ensemble-graph-ab288-7cb46799f-b5llf\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") " pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:50.867028 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:18:50.867010    2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-ab288-serving-cert: secret "ensemble-graph-ab288-serving-cert" not found
Sep 17 22:18:50.867104 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:18:50.867092    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls podName:b006ed1e-3b9d-4974-b09a-e5025a0a6268 nodeName:}" failed. No retries permitted until 2026-09-17 22:18:51.367071635 +0000 UTC m=+1262.181447004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls") pod "ensemble-graph-ab288-7cb46799f-b5llf" (UID: "b006ed1e-3b9d-4974-b09a-e5025a0a6268") : secret "ensemble-graph-ab288-serving-cert" not found
Sep 17 22:18:50.867671 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:50.867652    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b006ed1e-3b9d-4974-b09a-e5025a0a6268-openshift-service-ca-bundle\") pod \"ensemble-graph-ab288-7cb46799f-b5llf\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") " pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:51.133171 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:51.133086    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c23e2670-b897-425f-b746-f37006aab637" path="/var/lib/kubelet/pods/c23e2670-b897-425f-b746-f37006aab637/volumes"
Sep 17 22:18:51.371331 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:51.371294    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls\") pod \"ensemble-graph-ab288-7cb46799f-b5llf\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") " pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:51.373426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:51.373406    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls\") pod \"ensemble-graph-ab288-7cb46799f-b5llf\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") " pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:51.597692 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:51.597648    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:51.866850 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:51.866773    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"]
Sep 17 22:18:51.869633 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:18:51.869591    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb006ed1e_3b9d_4974_b09a_e5025a0a6268.slice/crio-25e8292fda50a1d54145a6149040bf0ea32721f57122a3f59acba26a61da6801 WatchSource:0}: Error finding container 25e8292fda50a1d54145a6149040bf0ea32721f57122a3f59acba26a61da6801: Status 404 returned error can't find the container with id 25e8292fda50a1d54145a6149040bf0ea32721f57122a3f59acba26a61da6801
Sep 17 22:18:52.489661 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:52.489624    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" event={"ID":"b006ed1e-3b9d-4974-b09a-e5025a0a6268","Type":"ContainerStarted","Data":"16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2"}
Sep 17 22:18:52.489856 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:52.489682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" event={"ID":"b006ed1e-3b9d-4974-b09a-e5025a0a6268","Type":"ContainerStarted","Data":"25e8292fda50a1d54145a6149040bf0ea32721f57122a3f59acba26a61da6801"}
Sep 17 22:18:52.489856 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:52.489768    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:18:52.504134 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:52.504091    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podStartSLOduration=2.504077105 podStartE2EDuration="2.504077105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:18:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:18:52.503075199 +0000 UTC m=+1263.317450681" watchObservedRunningTime="2026-09-17 22:18:52.504077105 +0000 UTC m=+1263.318452481"
Sep 17 22:18:54.476771 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:54.476741    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:18:54.477291 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:54.477240    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 22:18:55.353197 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:55.353158    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:18:58.499897 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:18:58.499866    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:19:00.353623 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:00.353582    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:00.353962 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:00.353701    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:19:00.757473 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:00.757436    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"]
Sep 17 22:19:00.757756 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:00.757703    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288" containerID="cri-o://16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2" gracePeriod=30
Sep 17 22:19:00.867574 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:00.867541    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"]
Sep 17 22:19:00.867860 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:00.867826    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container" containerID="cri-o://3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09" gracePeriod=30
Sep 17 22:19:00.868090 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:00.867895    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kube-rbac-proxy" containerID="cri-o://6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a" gracePeriod=30
Sep 17 22:19:01.071429 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.071350    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"]
Sep 17 22:19:01.074771 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.074746    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.077395 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.077373    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1a2f0-kube-rbac-proxy-sar-config\""
Sep 17 22:19:01.077522 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.077412    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1a2f0-predictor-serving-cert\""
Sep 17 22:19:01.084804 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.084773    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"]
Sep 17 22:19:01.149332 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.149302    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2stk\" (UniqueName: \"kubernetes.io/projected/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-api-access-z2stk\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.149472 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.149348    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00b14ba9-258a-44e4-90a3-82cc3d39db76-proxy-tls\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.149526 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.149467    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.250125 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.250093    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2stk\" (UniqueName: \"kubernetes.io/projected/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-api-access-z2stk\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.250331 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.250143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00b14ba9-258a-44e4-90a3-82cc3d39db76-proxy-tls\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.250414 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.250335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.250955 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.250928    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.252782 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.252761    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00b14ba9-258a-44e4-90a3-82cc3d39db76-proxy-tls\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.257829 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.257806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2stk\" (UniqueName: \"kubernetes.io/projected/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-api-access-z2stk\") pod \"error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") " pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.386809 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.386733    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:01.532376 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.532340    2561 generic.go:356] "Generic (PLEG): container finished" podID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerID="6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a" exitCode=2
Sep 17 22:19:01.532544 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.532414    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" event={"ID":"50d0de5a-be87-4994-9c37-42800d1c7f61","Type":"ContainerDied","Data":"6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a"}
Sep 17 22:19:01.658308 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:01.658276    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"]
Sep 17 22:19:01.661032 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:19:01.661010    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b14ba9_258a_44e4_90a3_82cc3d39db76.slice/crio-f2628888ada863d168013142d2e409e11ba16aff46615bf8f00a5fc00f514188 WatchSource:0}: Error finding container f2628888ada863d168013142d2e409e11ba16aff46615bf8f00a5fc00f514188: Status 404 returned error can't find the container with id f2628888ada863d168013142d2e409e11ba16aff46615bf8f00a5fc00f514188
Sep 17 22:19:02.538662 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:02.538628    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" event={"ID":"00b14ba9-258a-44e4-90a3-82cc3d39db76","Type":"ContainerStarted","Data":"84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e"}
Sep 17 22:19:02.539060 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:02.538692    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:02.539060 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:02.538713    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" event={"ID":"00b14ba9-258a-44e4-90a3-82cc3d39db76","Type":"ContainerStarted","Data":"f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015"}
Sep 17 22:19:02.539060 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:02.538731    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" event={"ID":"00b14ba9-258a-44e4-90a3-82cc3d39db76","Type":"ContainerStarted","Data":"f2628888ada863d168013142d2e409e11ba16aff46615bf8f00a5fc00f514188"}
Sep 17 22:19:02.539060 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:02.538835    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:02.540239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:02.540210    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 22:19:02.558567 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:02.558519    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podStartSLOduration=1.558504117 podStartE2EDuration="1.558504117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:19:02.557088999 +0000 UTC m=+1273.371464380" watchObservedRunningTime="2026-09-17 22:19:02.558504117 +0000 UTC m=+1273.372879493"
Sep 17 22:19:03.217529 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:03.217488    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.37:8643/healthz\": dial tcp 10.133.0.37:8643: connect: connection refused"
Sep 17 22:19:03.498494 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:03.498408    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:03.543133 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:03.543094    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 22:19:04.020187 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.020166    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:19:04.175064 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.175029    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qv4b\" (UniqueName: \"kubernetes.io/projected/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-api-access-7qv4b\") pod \"50d0de5a-be87-4994-9c37-42800d1c7f61\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") "
Sep 17 22:19:04.175284 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.175118    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-rbac-proxy-sar-config\") pod \"50d0de5a-be87-4994-9c37-42800d1c7f61\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") "
Sep 17 22:19:04.175284 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.175166    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50d0de5a-be87-4994-9c37-42800d1c7f61-proxy-tls\") pod \"50d0de5a-be87-4994-9c37-42800d1c7f61\" (UID: \"50d0de5a-be87-4994-9c37-42800d1c7f61\") "
Sep 17 22:19:04.175555 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.175527    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "50d0de5a-be87-4994-9c37-42800d1c7f61" (UID: "50d0de5a-be87-4994-9c37-42800d1c7f61"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:19:04.177220 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.177196    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-api-access-7qv4b" (OuterVolumeSpecName: "kube-api-access-7qv4b") pod "50d0de5a-be87-4994-9c37-42800d1c7f61" (UID: "50d0de5a-be87-4994-9c37-42800d1c7f61"). InnerVolumeSpecName "kube-api-access-7qv4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:19:04.177348 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.177228    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d0de5a-be87-4994-9c37-42800d1c7f61-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "50d0de5a-be87-4994-9c37-42800d1c7f61" (UID: "50d0de5a-be87-4994-9c37-42800d1c7f61"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:19:04.276751 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.276720    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50d0de5a-be87-4994-9c37-42800d1c7f61-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:19:04.276751 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.276743    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qv4b\" (UniqueName: \"kubernetes.io/projected/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-api-access-7qv4b\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:19:04.276751 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.276753    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50d0de5a-be87-4994-9c37-42800d1c7f61-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:19:04.478373 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.478275    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 22:19:04.548892 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.548860    2561 generic.go:356] "Generic (PLEG): container finished" podID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerID="3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09" exitCode=0
Sep 17 22:19:04.549270 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.548913    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"
Sep 17 22:19:04.549270 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.548939    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" event={"ID":"50d0de5a-be87-4994-9c37-42800d1c7f61","Type":"ContainerDied","Data":"3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09"}
Sep 17 22:19:04.549270 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.548995    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq" event={"ID":"50d0de5a-be87-4994-9c37-42800d1c7f61","Type":"ContainerDied","Data":"8c207771ad1173973af75e34898ebee11d1fe28d6f3ba2e7b46b946dc7772d13"}
Sep 17 22:19:04.549270 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.549022    2561 scope.go:117] "RemoveContainer" containerID="6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a"
Sep 17 22:19:04.557357 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.557340    2561 scope.go:117] "RemoveContainer" containerID="3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09"
Sep 17 22:19:04.564592 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.564573    2561 scope.go:117] "RemoveContainer" containerID="6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a"
Sep 17 22:19:04.564847 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:19:04.564828    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a\": container with ID starting with 6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a not found: ID does not exist" containerID="6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a"
Sep 17 22:19:04.564900 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.564855    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a"} err="failed to get container status \"6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a\": rpc error: code = NotFound desc = could not find container \"6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a\": container with ID starting with 6318807babd2cb3135db858ec5c965f89fd534e8a4198ba7183033d6cd32548a not found: ID does not exist"
Sep 17 22:19:04.564900 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.564871    2561 scope.go:117] "RemoveContainer" containerID="3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09"
Sep 17 22:19:04.565134 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:19:04.565117    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09\": container with ID starting with 3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09 not found: ID does not exist" containerID="3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09"
Sep 17 22:19:04.565187 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.565139    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09"} err="failed to get container status \"3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09\": rpc error: code = NotFound desc = could not find container \"3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09\": container with ID starting with 3b3f11b0f2b3ab6b81947d0eae68cb465853de9271ca391b91b113abb3414c09 not found: ID does not exist"
Sep 17 22:19:04.569141 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.569122    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"]
Sep 17 22:19:04.571495 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:04.571479    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab288-predictor-c9f568f7f-nbvxq"]
Sep 17 22:19:05.132058 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:05.132029    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" path="/var/lib/kubelet/pods/50d0de5a-be87-4994-9c37-42800d1c7f61/volumes"
Sep 17 22:19:05.353812 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:05.353771    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:08.498723 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:08.498686    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:08.547462 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:08.547435    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:08.547840 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:08.547812    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 22:19:10.353020 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:10.352980    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:13.498892 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:13.498841    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:13.499330 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:13.498966    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:19:14.477417 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:14.477370    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 22:19:15.353931 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:15.353883    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:16.838482 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:16.838455    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:19:16.883725 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:16.883682    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b3180f-05fc-43d2-95bc-46cae51026eb-openshift-service-ca-bundle\") pod \"93b3180f-05fc-43d2-95bc-46cae51026eb\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") "
Sep 17 22:19:16.883874 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:16.883789    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls\") pod \"93b3180f-05fc-43d2-95bc-46cae51026eb\" (UID: \"93b3180f-05fc-43d2-95bc-46cae51026eb\") "
Sep 17 22:19:16.884004 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:16.883982    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93b3180f-05fc-43d2-95bc-46cae51026eb-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "93b3180f-05fc-43d2-95bc-46cae51026eb" (UID: "93b3180f-05fc-43d2-95bc-46cae51026eb"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:19:16.885795 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:16.885775    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "93b3180f-05fc-43d2-95bc-46cae51026eb" (UID: "93b3180f-05fc-43d2-95bc-46cae51026eb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:19:16.984889 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:16.984792    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b3180f-05fc-43d2-95bc-46cae51026eb-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:19:16.984889 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:16.984820    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/93b3180f-05fc-43d2-95bc-46cae51026eb-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:19:17.611791 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.611749    2561 generic.go:356] "Generic (PLEG): container finished" podID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerID="a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2" exitCode=0
Sep 17 22:19:17.611953 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.611804    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" event={"ID":"93b3180f-05fc-43d2-95bc-46cae51026eb","Type":"ContainerDied","Data":"a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2"}
Sep 17 22:19:17.611953 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.611863    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9" event={"ID":"93b3180f-05fc-43d2-95bc-46cae51026eb","Type":"ContainerDied","Data":"16e884254a97a37d1d820c014e5d4628720ee56a6d01dd8253a4b84e82592137"}
Sep 17 22:19:17.611953 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.611805    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"
Sep 17 22:19:17.611953 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.611883    2561 scope.go:117] "RemoveContainer" containerID="a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2"
Sep 17 22:19:17.619918 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.619898    2561 scope.go:117] "RemoveContainer" containerID="a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2"
Sep 17 22:19:17.620189 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:19:17.620169    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2\": container with ID starting with a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2 not found: ID does not exist" containerID="a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2"
Sep 17 22:19:17.620280 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.620195    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2"} err="failed to get container status \"a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2\": rpc error: code = NotFound desc = could not find container \"a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2\": container with ID starting with a3f149a5efa6a18f5a18ad8fa028fcdb5acc9091745902373cc5c83ef50b60e2 not found: ID does not exist"
Sep 17 22:19:17.627016 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.626994    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"]
Sep 17 22:19:17.630060 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:17.630038    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a4672-66f4cddcbb-qrgh9"]
Sep 17 22:19:18.498802 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:18.498767    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:18.548526 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:18.548489    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 22:19:19.132852 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:19.132815    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" path="/var/lib/kubelet/pods/93b3180f-05fc-43d2-95bc-46cae51026eb/volumes"
Sep 17 22:19:23.498188 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:23.498151    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:24.477785 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:24.477744    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 22:19:28.498692 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:28.498648    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:28.548211 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:28.548178    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 22:19:30.901498 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:30.901473    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:19:30.990018 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:30.989988    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b006ed1e-3b9d-4974-b09a-e5025a0a6268-openshift-service-ca-bundle\") pod \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") "
Sep 17 22:19:30.990146 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:30.990051    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls\") pod \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\" (UID: \"b006ed1e-3b9d-4974-b09a-e5025a0a6268\") "
Sep 17 22:19:30.990341 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:30.990320    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b006ed1e-3b9d-4974-b09a-e5025a0a6268-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b006ed1e-3b9d-4974-b09a-e5025a0a6268" (UID: "b006ed1e-3b9d-4974-b09a-e5025a0a6268"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:19:30.991982 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:30.991964    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b006ed1e-3b9d-4974-b09a-e5025a0a6268" (UID: "b006ed1e-3b9d-4974-b09a-e5025a0a6268"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:19:31.090730 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.090667    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b006ed1e-3b9d-4974-b09a-e5025a0a6268-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:19:31.090730 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.090690    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b006ed1e-3b9d-4974-b09a-e5025a0a6268-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:19:31.673866 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.673833    2561 generic.go:356] "Generic (PLEG): container finished" podID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerID="16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2" exitCode=0
Sep 17 22:19:31.674053 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.673883    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"
Sep 17 22:19:31.674053 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.673884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" event={"ID":"b006ed1e-3b9d-4974-b09a-e5025a0a6268","Type":"ContainerDied","Data":"16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2"}
Sep 17 22:19:31.674053 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.674028    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf" event={"ID":"b006ed1e-3b9d-4974-b09a-e5025a0a6268","Type":"ContainerDied","Data":"25e8292fda50a1d54145a6149040bf0ea32721f57122a3f59acba26a61da6801"}
Sep 17 22:19:31.674217 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.674065    2561 scope.go:117] "RemoveContainer" containerID="16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2"
Sep 17 22:19:31.682701 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.682676    2561 scope.go:117] "RemoveContainer" containerID="16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2"
Sep 17 22:19:31.682980 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:19:31.682949    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2\": container with ID starting with 16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2 not found: ID does not exist" containerID="16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2"
Sep 17 22:19:31.683081 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.682979    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2"} err="failed to get container status \"16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2\": rpc error: code = NotFound desc = could not find container \"16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2\": container with ID starting with 16c7fe3a0976273700114f85b83d07480656c63244b742cc25c2e4567ee055c2 not found: ID does not exist"
Sep 17 22:19:31.688562 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.688543    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"]
Sep 17 22:19:31.692149 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:31.692129    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-ab288-7cb46799f-b5llf"]
Sep 17 22:19:33.132453 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:33.132422    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" path="/var/lib/kubelet/pods/b006ed1e-3b9d-4974-b09a-e5025a0a6268/volumes"
Sep 17 22:19:34.478129 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:34.478099    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:19:38.547916 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:38.547873    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 22:19:46.878430 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878398    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"]
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878737    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878748    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878757    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878763    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878771    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878777    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878793    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kube-rbac-proxy"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878798    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kube-rbac-proxy"
Sep 17 22:19:46.878849 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878845    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kserve-container"
Sep 17 22:19:46.879117 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878854    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b006ed1e-3b9d-4974-b09a-e5025a0a6268" containerName="ensemble-graph-ab288"
Sep 17 22:19:46.879117 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878861    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b3180f-05fc-43d2-95bc-46cae51026eb" containerName="sequence-graph-a4672"
Sep 17 22:19:46.879117 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.878868    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d0de5a-be87-4994-9c37-42800d1c7f61" containerName="kube-rbac-proxy"
Sep 17 22:19:46.881775 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.881758    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:46.883939 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.883915    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-1fd26-serving-cert\""
Sep 17 22:19:46.883939 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.883933    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-1fd26-kube-rbac-proxy-sar-config\""
Sep 17 22:19:46.890511 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.890488    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"]
Sep 17 22:19:46.906541 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.906520    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec11051-be28-4149-b08e-81565bb83ba8-openshift-service-ca-bundle\") pod \"sequence-graph-1fd26-5957b889bc-s9g2m\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") " pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:46.906650 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:46.906569    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls\") pod \"sequence-graph-1fd26-5957b889bc-s9g2m\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") " pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:47.007313 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:47.007289    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec11051-be28-4149-b08e-81565bb83ba8-openshift-service-ca-bundle\") pod \"sequence-graph-1fd26-5957b889bc-s9g2m\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") " pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:47.007449 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:47.007353    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls\") pod \"sequence-graph-1fd26-5957b889bc-s9g2m\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") " pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:47.007514 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:19:47.007469    2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-1fd26-serving-cert: secret "sequence-graph-1fd26-serving-cert" not found
Sep 17 22:19:47.007572 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:19:47.007549    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls podName:8ec11051-be28-4149-b08e-81565bb83ba8 nodeName:}" failed. No retries permitted until 2026-09-17 22:19:47.507527169 +0000 UTC m=+1318.321902543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls") pod "sequence-graph-1fd26-5957b889bc-s9g2m" (UID: "8ec11051-be28-4149-b08e-81565bb83ba8") : secret "sequence-graph-1fd26-serving-cert" not found
Sep 17 22:19:47.008054 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:47.008031    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec11051-be28-4149-b08e-81565bb83ba8-openshift-service-ca-bundle\") pod \"sequence-graph-1fd26-5957b889bc-s9g2m\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") " pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:47.511036 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:47.511005    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls\") pod \"sequence-graph-1fd26-5957b889bc-s9g2m\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") " pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:47.513281 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:47.513262    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls\") pod \"sequence-graph-1fd26-5957b889bc-s9g2m\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") " pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:47.793033 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:47.792953    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:48.062357 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:48.062332    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"]
Sep 17 22:19:48.064525 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:19:48.064494    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec11051_be28_4149_b08e_81565bb83ba8.slice/crio-b01b76a18e26b5af53972d513b43befb61e0e838c5ce574f1ea5a39521b42a20 WatchSource:0}: Error finding container b01b76a18e26b5af53972d513b43befb61e0e838c5ce574f1ea5a39521b42a20: Status 404 returned error can't find the container with id b01b76a18e26b5af53972d513b43befb61e0e838c5ce574f1ea5a39521b42a20
Sep 17 22:19:48.549435 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:48.549407    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:19:48.749177 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:48.749144    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" event={"ID":"8ec11051-be28-4149-b08e-81565bb83ba8","Type":"ContainerStarted","Data":"1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1"}
Sep 17 22:19:48.749370 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:48.749192    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" event={"ID":"8ec11051-be28-4149-b08e-81565bb83ba8","Type":"ContainerStarted","Data":"b01b76a18e26b5af53972d513b43befb61e0e838c5ce574f1ea5a39521b42a20"}
Sep 17 22:19:48.749370 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:48.749290    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:48.765731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:48.765686    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podStartSLOduration=2.7656711449999998 podStartE2EDuration="2.765671145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:19:48.763991871 +0000 UTC m=+1319.578367247" watchObservedRunningTime="2026-09-17 22:19:48.765671145 +0000 UTC m=+1319.580046547"
Sep 17 22:19:54.759125 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:54.759094    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:19:56.964536 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:56.964495    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"]
Sep 17 22:19:56.964984 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:56.964719    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26" containerID="cri-o://1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1" gracePeriod=30
Sep 17 22:19:57.063380 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.063349    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"]
Sep 17 22:19:57.063650 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.063610    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container" containerID="cri-o://7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8" gracePeriod=30
Sep 17 22:19:57.063778 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.063649    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kube-rbac-proxy" containerID="cri-o://296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c" gracePeriod=30
Sep 17 22:19:57.234809 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.234742    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"]
Sep 17 22:19:57.238063 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.238048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.240526 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.240504    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6bcee-kube-rbac-proxy-sar-config\""
Sep 17 22:19:57.240666 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.240504    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6bcee-predictor-serving-cert\""
Sep 17 22:19:57.246456 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.246208    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"]
Sep 17 22:19:57.287026 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.287005    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-proxy-tls\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.287134 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.287033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrzf9\" (UniqueName: \"kubernetes.io/projected/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-api-access-mrzf9\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.287134 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.287097    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.388152 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.388117    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.388365 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.388179    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-proxy-tls\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.388365 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.388203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrzf9\" (UniqueName: \"kubernetes.io/projected/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-api-access-mrzf9\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.388838 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.388821    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.390456 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.390437    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-proxy-tls\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.395397 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.395374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrzf9\" (UniqueName: \"kubernetes.io/projected/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-api-access-mrzf9\") pod \"error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") " pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.549941 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.549856    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:57.790872 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.790841    2561 generic.go:356] "Generic (PLEG): container finished" podID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerID="296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c" exitCode=2
Sep 17 22:19:57.791037 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.790912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" event={"ID":"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e","Type":"ContainerDied","Data":"296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c"}
Sep 17 22:19:57.818439 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:57.818417    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"]
Sep 17 22:19:57.820295 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:19:57.820269    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d06a762_c3dc_42a2_8c3a_3ac1352b1f39.slice/crio-214dc2c50696f104c847442fe810e5ba4bc33f3db8ac314593722b66591d7b4a WatchSource:0}: Error finding container 214dc2c50696f104c847442fe810e5ba4bc33f3db8ac314593722b66591d7b4a: Status 404 returned error can't find the container with id 214dc2c50696f104c847442fe810e5ba4bc33f3db8ac314593722b66591d7b4a
Sep 17 22:19:58.798056 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:58.798023    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" event={"ID":"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39","Type":"ContainerStarted","Data":"58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4"}
Sep 17 22:19:58.798502 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:58.798084    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" event={"ID":"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39","Type":"ContainerStarted","Data":"a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e"}
Sep 17 22:19:58.798502 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:58.798106    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" event={"ID":"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39","Type":"ContainerStarted","Data":"214dc2c50696f104c847442fe810e5ba4bc33f3db8ac314593722b66591d7b4a"}
Sep 17 22:19:58.798502 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:58.798262    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:58.798502 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:58.798331    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:19:58.799631 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:58.799601    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 22:19:58.814876 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:58.814834    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podStartSLOduration=1.814821824 podStartE2EDuration="1.814821824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:19:58.814062685 +0000 UTC m=+1329.628438073" watchObservedRunningTime="2026-09-17 22:19:58.814821824 +0000 UTC m=+1329.629197200"
Sep 17 22:19:59.473212 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:59.473171    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.38:8643/healthz\": dial tcp 10.133.0.38:8643: connect: connection refused"
Sep 17 22:19:59.757847 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:59.757752    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:19:59.802762 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:19:59.802721    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 22:20:00.108130 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.108102    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:20:00.209265 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.209222    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls\") pod \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") "
Sep 17 22:20:00.209426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.209302    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq5mb\" (UniqueName: \"kubernetes.io/projected/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-api-access-cq5mb\") pod \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") "
Sep 17 22:20:00.209426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.209323    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-rbac-proxy-sar-config\") pod \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\" (UID: \"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e\") "
Sep 17 22:20:00.209643 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.209625    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" (UID: "4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:20:00.211414 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.211379    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" (UID: "4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:20:00.211414 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.211383    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-api-access-cq5mb" (OuterVolumeSpecName: "kube-api-access-cq5mb") pod "4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" (UID: "4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e"). InnerVolumeSpecName "kube-api-access-cq5mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:20:00.310112 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.310044    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq5mb\" (UniqueName: \"kubernetes.io/projected/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-api-access-cq5mb\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:20:00.310112 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.310070    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:20:00.310112 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.310081    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:20:00.808751 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.808721    2561 generic.go:356] "Generic (PLEG): container finished" podID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerID="7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8" exitCode=0
Sep 17 22:20:00.809164 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.808772    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"
Sep 17 22:20:00.809164 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.808788    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" event={"ID":"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e","Type":"ContainerDied","Data":"7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8"}
Sep 17 22:20:00.809164 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.808836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl" event={"ID":"4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e","Type":"ContainerDied","Data":"140e92f8e4461c4286db5a31748c52a3c3c6dfca215f4d514470661f4105b71a"}
Sep 17 22:20:00.809164 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.808863    2561 scope.go:117] "RemoveContainer" containerID="296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c"
Sep 17 22:20:00.817628 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.817613    2561 scope.go:117] "RemoveContainer" containerID="7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8"
Sep 17 22:20:00.824519 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.824502    2561 scope.go:117] "RemoveContainer" containerID="296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c"
Sep 17 22:20:00.824757 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:20:00.824741    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c\": container with ID starting with 296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c not found: ID does not exist" containerID="296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c"
Sep 17 22:20:00.824801 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.824764    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c"} err="failed to get container status \"296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c\": rpc error: code = NotFound desc = could not find container \"296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c\": container with ID starting with 296abe66c560e9edacf0b4016da25c24f491d5b27dc6e4b17610d61b11750f5c not found: ID does not exist"
Sep 17 22:20:00.824801 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.824779    2561 scope.go:117] "RemoveContainer" containerID="7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8"
Sep 17 22:20:00.825008 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:20:00.824993    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8\": container with ID starting with 7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8 not found: ID does not exist" containerID="7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8"
Sep 17 22:20:00.825053 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.825012    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8"} err="failed to get container status \"7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8\": rpc error: code = NotFound desc = could not find container \"7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8\": container with ID starting with 7d0160be7cc45c595fe17c6bb10270ad1c467fda5e2dfa253bfc7528997c46e8 not found: ID does not exist"
Sep 17 22:20:00.828590 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.828568    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"]
Sep 17 22:20:00.833786 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.833765    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1fd26-predictor-64d658b8f5-b29nl"]
Sep 17 22:20:00.961761 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.961733    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"]
Sep 17 22:20:00.962115 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.962097    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kube-rbac-proxy"
Sep 17 22:20:00.962191 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.962117    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kube-rbac-proxy"
Sep 17 22:20:00.962191 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.962148    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container"
Sep 17 22:20:00.962191 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.962158    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container"
Sep 17 22:20:00.962367 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.962227    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kube-rbac-proxy"
Sep 17 22:20:00.962367 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.962242    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" containerName="kserve-container"
Sep 17 22:20:00.966378 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.966360    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:00.968520 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.968498    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1a2f0-serving-cert\""
Sep 17 22:20:00.968520 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.968512    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1a2f0-kube-rbac-proxy-sar-config\""
Sep 17 22:20:00.977679 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:00.976371    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"]
Sep 17 22:20:01.015102 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.015072    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-openshift-service-ca-bundle\") pod \"ensemble-graph-1a2f0-f7b64778c-6jwp5\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") " pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.015102 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.015104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-proxy-tls\") pod \"ensemble-graph-1a2f0-f7b64778c-6jwp5\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") " pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.115494 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.115426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-openshift-service-ca-bundle\") pod \"ensemble-graph-1a2f0-f7b64778c-6jwp5\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") " pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.115494 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.115457    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-proxy-tls\") pod \"ensemble-graph-1a2f0-f7b64778c-6jwp5\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") " pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.115994 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.115971    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-openshift-service-ca-bundle\") pod \"ensemble-graph-1a2f0-f7b64778c-6jwp5\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") " pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.117782 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.117767    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-proxy-tls\") pod \"ensemble-graph-1a2f0-f7b64778c-6jwp5\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") " pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.132896 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.132870    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e" path="/var/lib/kubelet/pods/4e51baa5-18e4-4d6d-9fd8-52d510c3dc5e/volumes"
Sep 17 22:20:01.281654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.281625    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.551741 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.551620    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"]
Sep 17 22:20:01.553959 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:20:01.553933    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc466bf0a_bfe9_4df2_ab03_39bf0c40cf0e.slice/crio-c74bd21286441df5f98152fc3a73bb9831b0d22f1a56dce6c1e8c1ced542b2a7 WatchSource:0}: Error finding container c74bd21286441df5f98152fc3a73bb9831b0d22f1a56dce6c1e8c1ced542b2a7: Status 404 returned error can't find the container with id c74bd21286441df5f98152fc3a73bb9831b0d22f1a56dce6c1e8c1ced542b2a7
Sep 17 22:20:01.815879 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.815788    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" event={"ID":"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e","Type":"ContainerStarted","Data":"f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45"}
Sep 17 22:20:01.815879 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.815843    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" event={"ID":"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e","Type":"ContainerStarted","Data":"c74bd21286441df5f98152fc3a73bb9831b0d22f1a56dce6c1e8c1ced542b2a7"}
Sep 17 22:20:01.816369 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.815932    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:01.830823 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:01.830778    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podStartSLOduration=1.830766014 podStartE2EDuration="1.830766014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:20:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:20:01.82930451 +0000 UTC m=+1332.643679886" watchObservedRunningTime="2026-09-17 22:20:01.830766014 +0000 UTC m=+1332.645141391"
Sep 17 22:20:04.757111 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:04.757035    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:20:04.807313 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:04.807288    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:20:04.807690 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:04.807667    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 22:20:07.825507 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:07.825478    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:20:09.757173 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:09.757130    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:20:09.757598 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:09.757281    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:20:14.757369 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:14.757323    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:20:14.807781 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:14.807738    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 22:20:19.757732 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:19.757690    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:20:24.757064 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:24.757018    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:20:24.807884 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:24.807847    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 22:20:27.107962 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.107934    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:20:27.220530 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.220453    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec11051-be28-4149-b08e-81565bb83ba8-openshift-service-ca-bundle\") pod \"8ec11051-be28-4149-b08e-81565bb83ba8\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") "
Sep 17 22:20:27.220530 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.220517    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls\") pod \"8ec11051-be28-4149-b08e-81565bb83ba8\" (UID: \"8ec11051-be28-4149-b08e-81565bb83ba8\") "
Sep 17 22:20:27.220805 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.220782    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ec11051-be28-4149-b08e-81565bb83ba8-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "8ec11051-be28-4149-b08e-81565bb83ba8" (UID: "8ec11051-be28-4149-b08e-81565bb83ba8"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:20:27.222514 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.222496    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8ec11051-be28-4149-b08e-81565bb83ba8" (UID: "8ec11051-be28-4149-b08e-81565bb83ba8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:20:27.321183 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.321153    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec11051-be28-4149-b08e-81565bb83ba8-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:20:27.321183 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.321177    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ec11051-be28-4149-b08e-81565bb83ba8-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:20:27.934070 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.934036    2561 generic.go:356] "Generic (PLEG): container finished" podID="8ec11051-be28-4149-b08e-81565bb83ba8" containerID="1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1" exitCode=0
Sep 17 22:20:27.934239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.934078    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"
Sep 17 22:20:27.934239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.934113    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" event={"ID":"8ec11051-be28-4149-b08e-81565bb83ba8","Type":"ContainerDied","Data":"1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1"}
Sep 17 22:20:27.934239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.934163    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m" event={"ID":"8ec11051-be28-4149-b08e-81565bb83ba8","Type":"ContainerDied","Data":"b01b76a18e26b5af53972d513b43befb61e0e838c5ce574f1ea5a39521b42a20"}
Sep 17 22:20:27.934239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.934185    2561 scope.go:117] "RemoveContainer" containerID="1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1"
Sep 17 22:20:27.944594 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.944577    2561 scope.go:117] "RemoveContainer" containerID="1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1"
Sep 17 22:20:27.944856 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:20:27.944839    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1\": container with ID starting with 1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1 not found: ID does not exist" containerID="1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1"
Sep 17 22:20:27.944906 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.944866    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1"} err="failed to get container status \"1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1\": rpc error: code = NotFound desc = could not find container \"1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1\": container with ID starting with 1c45ab73daf88fb81408fb4546c444a52e5a0edeca5c503eb7ff4087f7c1bdb1 not found: ID does not exist"
Sep 17 22:20:27.956194 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.956170    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"]
Sep 17 22:20:27.959618 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:27.959596    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1fd26-5957b889bc-s9g2m"]
Sep 17 22:20:29.132451 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:29.132420    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" path="/var/lib/kubelet/pods/8ec11051-be28-4149-b08e-81565bb83ba8/volumes"
Sep 17 22:20:34.808697 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:34.808647    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 22:20:44.808421 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:44.808388    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:20:57.214336 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.214304    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"]
Sep 17 22:20:57.214731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.214643    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26"
Sep 17 22:20:57.214731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.214656    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26"
Sep 17 22:20:57.214731 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.214709    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec11051-be28-4149-b08e-81565bb83ba8" containerName="sequence-graph-1fd26"
Sep 17 22:20:57.218834 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.218812    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.221481 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.221450    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-6bcee-kube-rbac-proxy-sar-config\""
Sep 17 22:20:57.221605 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.221533    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-6bcee-serving-cert\""
Sep 17 22:20:57.234023 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.233997    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"]
Sep 17 22:20:57.322867 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.322838    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11cd1f95-1116-4cb7-9b48-0be92dc38471-proxy-tls\") pod \"sequence-graph-6bcee-76d5754-t957d\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") " pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.322986 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.322891    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11cd1f95-1116-4cb7-9b48-0be92dc38471-openshift-service-ca-bundle\") pod \"sequence-graph-6bcee-76d5754-t957d\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") " pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.423521 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.423495    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11cd1f95-1116-4cb7-9b48-0be92dc38471-proxy-tls\") pod \"sequence-graph-6bcee-76d5754-t957d\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") " pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.423636 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.423541    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11cd1f95-1116-4cb7-9b48-0be92dc38471-openshift-service-ca-bundle\") pod \"sequence-graph-6bcee-76d5754-t957d\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") " pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.424071 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.424053    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11cd1f95-1116-4cb7-9b48-0be92dc38471-openshift-service-ca-bundle\") pod \"sequence-graph-6bcee-76d5754-t957d\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") " pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.425872 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.425845    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11cd1f95-1116-4cb7-9b48-0be92dc38471-proxy-tls\") pod \"sequence-graph-6bcee-76d5754-t957d\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") " pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.528271 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.528196    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:57.796440 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:57.796418    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"]
Sep 17 22:20:57.799069 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:20:57.799041    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cd1f95_1116_4cb7_9b48_0be92dc38471.slice/crio-8666d4a28df91e84820dc4aaeb1b5bfb41a7588c0cd8aa12ccada3b8db5fd7c5 WatchSource:0}: Error finding container 8666d4a28df91e84820dc4aaeb1b5bfb41a7588c0cd8aa12ccada3b8db5fd7c5: Status 404 returned error can't find the container with id 8666d4a28df91e84820dc4aaeb1b5bfb41a7588c0cd8aa12ccada3b8db5fd7c5
Sep 17 22:20:58.069217 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:58.069124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" event={"ID":"11cd1f95-1116-4cb7-9b48-0be92dc38471","Type":"ContainerStarted","Data":"9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8"}
Sep 17 22:20:58.069217 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:58.069173    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" event={"ID":"11cd1f95-1116-4cb7-9b48-0be92dc38471","Type":"ContainerStarted","Data":"8666d4a28df91e84820dc4aaeb1b5bfb41a7588c0cd8aa12ccada3b8db5fd7c5"}
Sep 17 22:20:58.069424 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:58.069219    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:20:58.085409 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:20:58.085367    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podStartSLOduration=1.085354623 podStartE2EDuration="1.085354623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:20:58.082993291 +0000 UTC m=+1388.897368667" watchObservedRunningTime="2026-09-17 22:20:58.085354623 +0000 UTC m=+1388.899729996"
Sep 17 22:21:04.078610 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:21:04.078580    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:22:51.174439 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:22:51.174413    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:22:51.177733 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:22:51.177715    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:27:51.198789 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:27:51.198761    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:27:51.203282 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:27:51.203261    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:28:15.629791 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.629758    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"]
Sep 17 22:28:15.630494 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.630016    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0" containerID="cri-o://f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45" gracePeriod=30
Sep 17 22:28:15.776920 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.776888    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"]
Sep 17 22:28:15.777220 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.777185    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" containerID="cri-o://f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015" gracePeriod=30
Sep 17 22:28:15.777378 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.777242    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kube-rbac-proxy" containerID="cri-o://84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e" gracePeriod=30
Sep 17 22:28:15.916772 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.916734    2561 generic.go:356] "Generic (PLEG): container finished" podID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerID="84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e" exitCode=2
Sep 17 22:28:15.916946 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.916785    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" event={"ID":"00b14ba9-258a-44e4-90a3-82cc3d39db76","Type":"ContainerDied","Data":"84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e"}
Sep 17 22:28:15.953283 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.953239    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"]
Sep 17 22:28:15.956612 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.956590    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:15.959395 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.959376    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a2de1-predictor-serving-cert\""
Sep 17 22:28:15.959905 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.959888    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a2de1-kube-rbac-proxy-sar-config\""
Sep 17 22:28:15.974664 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:15.974645    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"]
Sep 17 22:28:16.011023 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.010996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.011183 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.011062    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.011183 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.011133    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtq6f\" (UniqueName: \"kubernetes.io/projected/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-api-access-wtq6f\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.111819 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.111791    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.111966 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.111829    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtq6f\" (UniqueName: \"kubernetes.io/projected/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-api-access-wtq6f\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.111966 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.111914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.111966 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:28:16.111937    2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-serving-cert: secret "error-404-isvc-a2de1-predictor-serving-cert" not found
Sep 17 22:28:16.112170 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:28:16.111996    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls podName:a354cd7e-013f-4e42-9120-6e9f9f3dc4ed nodeName:}" failed. No retries permitted until 2026-09-17 22:28:16.611978756 +0000 UTC m=+1827.426354123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls") pod "error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" (UID: "a354cd7e-013f-4e42-9120-6e9f9f3dc4ed") : secret "error-404-isvc-a2de1-predictor-serving-cert" not found
Sep 17 22:28:16.112601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.112580    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.123575 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.123550    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtq6f\" (UniqueName: \"kubernetes.io/projected/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-api-access-wtq6f\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.617231 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.617194    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.619584 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.619565    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls\") pod \"error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") " pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:16.867368 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:16.867270    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:17.136040 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:17.136019    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"]
Sep 17 22:28:17.138143 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:28:17.138115    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda354cd7e_013f_4e42_9120_6e9f9f3dc4ed.slice/crio-b7e3ff591e5fb64e24832ce669b50093d586992ed3313301690953d51cba74ce WatchSource:0}: Error finding container b7e3ff591e5fb64e24832ce669b50093d586992ed3313301690953d51cba74ce: Status 404 returned error can't find the container with id b7e3ff591e5fb64e24832ce669b50093d586992ed3313301690953d51cba74ce
Sep 17 22:28:17.824223 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:17.824187    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:28:17.927824 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:17.927789    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" event={"ID":"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed","Type":"ContainerStarted","Data":"b36dc75e2b6e99ea1b74aae92eaba6a2b7ce8a99a2b492c70d38340a03c7a6dc"}
Sep 17 22:28:17.928184 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:17.927851    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" event={"ID":"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed","Type":"ContainerStarted","Data":"5139d6007481ebb66d0ea2fdb7743c3417ab9e5cd42a3a09031d536b6c332b55"}
Sep 17 22:28:17.928184 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:17.927883    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:17.928184 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:17.927911    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" event={"ID":"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed","Type":"ContainerStarted","Data":"b7e3ff591e5fb64e24832ce669b50093d586992ed3313301690953d51cba74ce"}
Sep 17 22:28:17.944605 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:17.944559    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podStartSLOduration=2.944541839 podStartE2EDuration="2.944541839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:28:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:28:17.942905757 +0000 UTC m=+1828.757281133" watchObservedRunningTime="2026-09-17 22:28:17.944541839 +0000 UTC m=+1828.758917215"
Sep 17 22:28:18.543680 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.543640    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Sep 17 22:28:18.547950 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.547927    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 22:28:18.727973 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.727946    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:28:18.834191 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.834109    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2stk\" (UniqueName: \"kubernetes.io/projected/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-api-access-z2stk\") pod \"00b14ba9-258a-44e4-90a3-82cc3d39db76\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") "
Sep 17 22:28:18.834191 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.834151    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00b14ba9-258a-44e4-90a3-82cc3d39db76-proxy-tls\") pod \"00b14ba9-258a-44e4-90a3-82cc3d39db76\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") "
Sep 17 22:28:18.834515 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.834208    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-rbac-proxy-sar-config\") pod \"00b14ba9-258a-44e4-90a3-82cc3d39db76\" (UID: \"00b14ba9-258a-44e4-90a3-82cc3d39db76\") "
Sep 17 22:28:18.834581 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.834546    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "00b14ba9-258a-44e4-90a3-82cc3d39db76" (UID: "00b14ba9-258a-44e4-90a3-82cc3d39db76"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:28:18.836261 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.836234    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b14ba9-258a-44e4-90a3-82cc3d39db76-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "00b14ba9-258a-44e4-90a3-82cc3d39db76" (UID: "00b14ba9-258a-44e4-90a3-82cc3d39db76"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:28:18.836366 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.836343    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-api-access-z2stk" (OuterVolumeSpecName: "kube-api-access-z2stk") pod "00b14ba9-258a-44e4-90a3-82cc3d39db76" (UID: "00b14ba9-258a-44e4-90a3-82cc3d39db76"). InnerVolumeSpecName "kube-api-access-z2stk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:28:18.933783 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.933751    2561 generic.go:356] "Generic (PLEG): container finished" podID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerID="f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015" exitCode=0
Sep 17 22:28:18.934116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.933806    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"
Sep 17 22:28:18.934116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.933835    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" event={"ID":"00b14ba9-258a-44e4-90a3-82cc3d39db76","Type":"ContainerDied","Data":"f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015"}
Sep 17 22:28:18.934116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.933943    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs" event={"ID":"00b14ba9-258a-44e4-90a3-82cc3d39db76","Type":"ContainerDied","Data":"f2628888ada863d168013142d2e409e11ba16aff46615bf8f00a5fc00f514188"}
Sep 17 22:28:18.934116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.933978    2561 scope.go:117] "RemoveContainer" containerID="84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e"
Sep 17 22:28:18.934450 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.934429    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:18.934771 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.934748    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2stk\" (UniqueName: \"kubernetes.io/projected/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-api-access-z2stk\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:28:18.934771 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.934772    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00b14ba9-258a-44e4-90a3-82cc3d39db76-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:28:18.934771 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.934783    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00b14ba9-258a-44e4-90a3-82cc3d39db76-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:28:18.935823 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.935790    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 22:28:18.945217 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.945189    2561 scope.go:117] "RemoveContainer" containerID="f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015"
Sep 17 22:28:18.952824 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.952807    2561 scope.go:117] "RemoveContainer" containerID="84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e"
Sep 17 22:28:18.953075 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:28:18.953057    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e\": container with ID starting with 84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e not found: ID does not exist" containerID="84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e"
Sep 17 22:28:18.953125 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.953083    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e"} err="failed to get container status \"84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e\": rpc error: code = NotFound desc = could not find container \"84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e\": container with ID starting with 84adbadef4cc65d2fdca80842c915e2ac7b657ab754949a380db3a4a1cece51e not found: ID does not exist"
Sep 17 22:28:18.953125 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.953099    2561 scope.go:117] "RemoveContainer" containerID="f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015"
Sep 17 22:28:18.953376 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:28:18.953358    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015\": container with ID starting with f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015 not found: ID does not exist" containerID="f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015"
Sep 17 22:28:18.953422 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.953381    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015"} err="failed to get container status \"f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015\": rpc error: code = NotFound desc = could not find container \"f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015\": container with ID starting with f76404105725548eb1d5f0c1613c6dc68214c34592a6faa5fd770f38aec4b015 not found: ID does not exist"
Sep 17 22:28:18.956374 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.956355    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"]
Sep 17 22:28:18.959444 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:18.959425    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1a2f0-predictor-f769ccbfc-q25hs"]
Sep 17 22:28:19.132451 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:19.132382    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" path="/var/lib/kubelet/pods/00b14ba9-258a-44e4-90a3-82cc3d39db76/volumes"
Sep 17 22:28:19.938962 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:19.938925    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 22:28:22.824655 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:22.824610    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:28:24.943647 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:24.943619    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:28:24.944154 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:24.944124    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 22:28:27.823613 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:27.823579    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:28:27.824054 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:27.823675    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:28:32.824327 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:32.824291    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:28:34.944680 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:34.944640    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 22:28:37.824363 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:37.824326    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:28:42.824417 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:42.824379    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:28:44.944570 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:44.944530    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 22:28:45.775376 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:45.775353    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:28:45.947469 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:45.947444    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-proxy-tls\") pod \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") "
Sep 17 22:28:45.947822 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:45.947510    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-openshift-service-ca-bundle\") pod \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\" (UID: \"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e\") "
Sep 17 22:28:45.947822 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:45.947788    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" (UID: "c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:28:45.949455 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:45.949416    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" (UID: "c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:28:46.048112 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.048083    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:28:46.048112 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.048107    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:28:46.052630 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.052604    2561 generic.go:356] "Generic (PLEG): container finished" podID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerID="f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45" exitCode=0
Sep 17 22:28:46.052754 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.052640    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" event={"ID":"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e","Type":"ContainerDied","Data":"f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45"}
Sep 17 22:28:46.052754 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.052656    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"
Sep 17 22:28:46.052754 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.052705    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5" event={"ID":"c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e","Type":"ContainerDied","Data":"c74bd21286441df5f98152fc3a73bb9831b0d22f1a56dce6c1e8c1ced542b2a7"}
Sep 17 22:28:46.052754 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.052726    2561 scope.go:117] "RemoveContainer" containerID="f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45"
Sep 17 22:28:46.061571 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.061553    2561 scope.go:117] "RemoveContainer" containerID="f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45"
Sep 17 22:28:46.061837 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:28:46.061817    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45\": container with ID starting with f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45 not found: ID does not exist" containerID="f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45"
Sep 17 22:28:46.061900 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.061844    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45"} err="failed to get container status \"f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45\": rpc error: code = NotFound desc = could not find container \"f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45\": container with ID starting with f5ec24f1542595342b35cd43d26fd451eb09285fce69f4c3ac61b424b2559a45 not found: ID does not exist"
Sep 17 22:28:46.072593 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.072567    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"]
Sep 17 22:28:46.075854 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:46.075835    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1a2f0-f7b64778c-6jwp5"]
Sep 17 22:28:47.133191 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:47.133155    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" path="/var/lib/kubelet/pods/c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e/volumes"
Sep 17 22:28:54.944964 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:28:54.944921    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 22:29:04.945956 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:04.945874    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:29:11.763179 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:11.763145    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"]
Sep 17 22:29:11.763589 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:11.763360    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee" containerID="cri-o://9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8" gracePeriod=30
Sep 17 22:29:11.882004 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:11.881973    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"]
Sep 17 22:29:11.882203 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:11.882177    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container" containerID="cri-o://a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e" gracePeriod=30
Sep 17 22:29:11.882313 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:11.882202    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kube-rbac-proxy" containerID="cri-o://58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4" gracePeriod=30
Sep 17 22:29:12.039011 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.038941    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"]
Sep 17 22:29:12.039398 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039383    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0"
Sep 17 22:29:12.039463 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039400    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0"
Sep 17 22:29:12.039463 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039421    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container"
Sep 17 22:29:12.039463 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039427    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container"
Sep 17 22:29:12.039463 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039436    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kube-rbac-proxy"
Sep 17 22:29:12.039463 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039443    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kube-rbac-proxy"
Sep 17 22:29:12.039620 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039490    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c466bf0a-bfe9-4df2-ab03-39bf0c40cf0e" containerName="ensemble-graph-1a2f0"
Sep 17 22:29:12.039620 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039501    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kube-rbac-proxy"
Sep 17 22:29:12.039620 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.039507    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b14ba9-258a-44e4-90a3-82cc3d39db76" containerName="kserve-container"
Sep 17 22:29:12.042412 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.042397    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.044550 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.044518    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-94ae3-predictor-serving-cert\""
Sep 17 22:29:12.044550 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.044540    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-94ae3-kube-rbac-proxy-sar-config\""
Sep 17 22:29:12.053851 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.052032    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"]
Sep 17 22:29:12.139988 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.139955    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjnpw\" (UniqueName: \"kubernetes.io/projected/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-api-access-xjnpw\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.140153 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.139993    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b182630b-eec4-4342-98e9-a5f0c3b5b478-proxy-tls\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.140153 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.140024    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.164223 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.164191    2561 generic.go:356] "Generic (PLEG): container finished" podID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerID="58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4" exitCode=2
Sep 17 22:29:12.164365 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.164286    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" event={"ID":"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39","Type":"ContainerDied","Data":"58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4"}
Sep 17 22:29:12.241095 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.241063    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjnpw\" (UniqueName: \"kubernetes.io/projected/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-api-access-xjnpw\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.241236 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.241103    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b182630b-eec4-4342-98e9-a5f0c3b5b478-proxy-tls\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.241340 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.241231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.241840 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.241819    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.243441 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.243423    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b182630b-eec4-4342-98e9-a5f0c3b5b478-proxy-tls\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.248314 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.248292    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjnpw\" (UniqueName: \"kubernetes.io/projected/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-api-access-xjnpw\") pod \"error-404-isvc-94ae3-predictor-7dd5ccf689-j665r\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") " pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.353489 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.353421    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:12.620177 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:12.620153    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"]
Sep 17 22:29:12.622500 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:29:12.622472    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb182630b_eec4_4342_98e9_a5f0c3b5b478.slice/crio-e7870973f15121609d884d661ce4c41db004de76bb3b5c046e28a64de5332f01 WatchSource:0}: Error finding container e7870973f15121609d884d661ce4c41db004de76bb3b5c046e28a64de5332f01: Status 404 returned error can't find the container with id e7870973f15121609d884d661ce4c41db004de76bb3b5c046e28a64de5332f01
Sep 17 22:29:13.170930 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:13.170891    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" event={"ID":"b182630b-eec4-4342-98e9-a5f0c3b5b478","Type":"ContainerStarted","Data":"8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18"}
Sep 17 22:29:13.171315 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:13.170960    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" event={"ID":"b182630b-eec4-4342-98e9-a5f0c3b5b478","Type":"ContainerStarted","Data":"538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310"}
Sep 17 22:29:13.171315 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:13.170999    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:13.171315 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:13.171032    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" event={"ID":"b182630b-eec4-4342-98e9-a5f0c3b5b478","Type":"ContainerStarted","Data":"e7870973f15121609d884d661ce4c41db004de76bb3b5c046e28a64de5332f01"}
Sep 17 22:29:13.189304 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:13.189237    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podStartSLOduration=1.1892234720000001 podStartE2EDuration="1.189223472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:29:13.188386655 +0000 UTC m=+1884.002762031" watchObservedRunningTime="2026-09-17 22:29:13.189223472 +0000 UTC m=+1884.003598850"
Sep 17 22:29:14.077769 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.077731    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:14.175508 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.175481    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:14.177034 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.176995    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 22:29:14.736518 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.736496    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:29:14.759390 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.759364    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-rbac-proxy-sar-config\") pod \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") "
Sep 17 22:29:14.759520 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.759401    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-proxy-tls\") pod \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") "
Sep 17 22:29:14.759520 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.759430    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrzf9\" (UniqueName: \"kubernetes.io/projected/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-api-access-mrzf9\") pod \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\" (UID: \"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39\") "
Sep 17 22:29:14.759762 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.759733    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" (UID: "8d06a762-c3dc-42a2-8c3a-3ac1352b1f39"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:29:14.761464 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.761438    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" (UID: "8d06a762-c3dc-42a2-8c3a-3ac1352b1f39"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:29:14.761543 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.761480    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-api-access-mrzf9" (OuterVolumeSpecName: "kube-api-access-mrzf9") pod "8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" (UID: "8d06a762-c3dc-42a2-8c3a-3ac1352b1f39"). InnerVolumeSpecName "kube-api-access-mrzf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:29:14.860288 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.860202    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:14.860288 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.860226    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:14.860288 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:14.860236    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrzf9\" (UniqueName: \"kubernetes.io/projected/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39-kube-api-access-mrzf9\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:15.181103 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.181073    2561 generic.go:356] "Generic (PLEG): container finished" podID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerID="a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e" exitCode=0
Sep 17 22:29:15.181516 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.181144    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"
Sep 17 22:29:15.181516 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.181155    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" event={"ID":"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39","Type":"ContainerDied","Data":"a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e"}
Sep 17 22:29:15.181516 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.181204    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h" event={"ID":"8d06a762-c3dc-42a2-8c3a-3ac1352b1f39","Type":"ContainerDied","Data":"214dc2c50696f104c847442fe810e5ba4bc33f3db8ac314593722b66591d7b4a"}
Sep 17 22:29:15.181516 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.181229    2561 scope.go:117] "RemoveContainer" containerID="58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4"
Sep 17 22:29:15.181775 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.181646    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 22:29:15.189601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.189585    2561 scope.go:117] "RemoveContainer" containerID="a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e"
Sep 17 22:29:15.196914 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.196886    2561 scope.go:117] "RemoveContainer" containerID="58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4"
Sep 17 22:29:15.197204 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:29:15.197186    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4\": container with ID starting with 58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4 not found: ID does not exist" containerID="58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4"
Sep 17 22:29:15.197302 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.197212    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4"} err="failed to get container status \"58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4\": rpc error: code = NotFound desc = could not find container \"58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4\": container with ID starting with 58f54f8b2d4acfe7211eb43604073ed3a11e08a30ea351bf6b378addebdbf6b4 not found: ID does not exist"
Sep 17 22:29:15.197302 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.197228    2561 scope.go:117] "RemoveContainer" containerID="a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e"
Sep 17 22:29:15.197626 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:29:15.197598    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e\": container with ID starting with a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e not found: ID does not exist" containerID="a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e"
Sep 17 22:29:15.197626 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.197631    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e"} err="failed to get container status \"a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e\": rpc error: code = NotFound desc = could not find container \"a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e\": container with ID starting with a4cd70c331de1d0d77c145aae136652a2f213e196c10254d394fefd7655b069e not found: ID does not exist"
Sep 17 22:29:15.198225 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.198203    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"]
Sep 17 22:29:15.199748 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.199726    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6bcee-predictor-85458cdc4c-p8f2h"]
Sep 17 22:29:15.875882 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.875851    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"]
Sep 17 22:29:15.876175 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.876164    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kube-rbac-proxy"
Sep 17 22:29:15.876216 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.876177    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kube-rbac-proxy"
Sep 17 22:29:15.876216 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.876188    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container"
Sep 17 22:29:15.876216 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.876193    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container"
Sep 17 22:29:15.876333 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.876278    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kserve-container"
Sep 17 22:29:15.876333 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.876287    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" containerName="kube-rbac-proxy"
Sep 17 22:29:15.880690 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.880664    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:15.882702 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.882675    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-a2de1-kube-rbac-proxy-sar-config\""
Sep 17 22:29:15.882803 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.882759    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-a2de1-serving-cert\""
Sep 17 22:29:15.889050 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.889031    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"]
Sep 17 22:29:15.970868 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.970839    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d593d-d974-4fb1-92c7-688a9501197e-proxy-tls\") pod \"splitter-graph-a2de1-748b4b9f7d-wrx95\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") " pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:15.971045 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:15.970890    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d09d593d-d974-4fb1-92c7-688a9501197e-openshift-service-ca-bundle\") pod \"splitter-graph-a2de1-748b4b9f7d-wrx95\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") " pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:16.071616 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:16.071583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d593d-d974-4fb1-92c7-688a9501197e-proxy-tls\") pod \"splitter-graph-a2de1-748b4b9f7d-wrx95\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") " pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:16.071758 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:16.071631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d09d593d-d974-4fb1-92c7-688a9501197e-openshift-service-ca-bundle\") pod \"splitter-graph-a2de1-748b4b9f7d-wrx95\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") " pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:16.072258 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:16.072228    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d09d593d-d974-4fb1-92c7-688a9501197e-openshift-service-ca-bundle\") pod \"splitter-graph-a2de1-748b4b9f7d-wrx95\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") " pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:16.073998 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:16.073981    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d593d-d974-4fb1-92c7-688a9501197e-proxy-tls\") pod \"splitter-graph-a2de1-748b4b9f7d-wrx95\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") " pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:16.190604 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:16.190576    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:16.458545 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:16.458519    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"]
Sep 17 22:29:16.461100 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:29:16.461070    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09d593d_d974_4fb1_92c7_688a9501197e.slice/crio-44044955329d912b7483a02327476628f2058647bd22f7ff82db004f098fd354 WatchSource:0}: Error finding container 44044955329d912b7483a02327476628f2058647bd22f7ff82db004f098fd354: Status 404 returned error can't find the container with id 44044955329d912b7483a02327476628f2058647bd22f7ff82db004f098fd354
Sep 17 22:29:17.133100 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:17.133063    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d06a762-c3dc-42a2-8c3a-3ac1352b1f39" path="/var/lib/kubelet/pods/8d06a762-c3dc-42a2-8c3a-3ac1352b1f39/volumes"
Sep 17 22:29:17.191904 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:17.191873    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" event={"ID":"d09d593d-d974-4fb1-92c7-688a9501197e","Type":"ContainerStarted","Data":"e0b0bc96ab423e2ff105d95bb183f2a29d299f706e3bde40909a68b9e7e6b10e"}
Sep 17 22:29:17.192240 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:17.191929    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:17.192240 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:17.191945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" event={"ID":"d09d593d-d974-4fb1-92c7-688a9501197e","Type":"ContainerStarted","Data":"44044955329d912b7483a02327476628f2058647bd22f7ff82db004f098fd354"}
Sep 17 22:29:17.207492 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:17.207447    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podStartSLOduration=2.207433315 podStartE2EDuration="2.207433315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:29:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:29:17.207083974 +0000 UTC m=+1888.021459364" watchObservedRunningTime="2026-09-17 22:29:17.207433315 +0000 UTC m=+1888.021808691"
Sep 17 22:29:19.077945 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:19.077900    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:20.186638 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:20.186611    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:29:20.187154 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:20.187126    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 22:29:23.202023 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:23.201997    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:24.077484 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:24.077442    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:24.077658 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:24.077542    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:29:25.954738 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:25.954707    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"]
Sep 17 22:29:25.955172 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:25.954922    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1" containerID="cri-o://e0b0bc96ab423e2ff105d95bb183f2a29d299f706e3bde40909a68b9e7e6b10e" gracePeriod=30
Sep 17 22:29:26.075117 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.075079    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"]
Sep 17 22:29:26.075496 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.075370    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container" containerID="cri-o://5139d6007481ebb66d0ea2fdb7743c3417ab9e5cd42a3a09031d536b6c332b55" gracePeriod=30
Sep 17 22:29:26.075496 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.075406    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kube-rbac-proxy" containerID="cri-o://b36dc75e2b6e99ea1b74aae92eaba6a2b7ce8a99a2b492c70d38340a03c7a6dc" gracePeriod=30
Sep 17 22:29:26.231754 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.231681    2561 generic.go:356] "Generic (PLEG): container finished" podID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerID="b36dc75e2b6e99ea1b74aae92eaba6a2b7ce8a99a2b492c70d38340a03c7a6dc" exitCode=2
Sep 17 22:29:26.231936 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.231749    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" event={"ID":"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed","Type":"ContainerDied","Data":"b36dc75e2b6e99ea1b74aae92eaba6a2b7ce8a99a2b492c70d38340a03c7a6dc"}
Sep 17 22:29:26.247827 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.247801    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"]
Sep 17 22:29:26.251138 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.251122    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.253543 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.253513    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0bca3-predictor-serving-cert\""
Sep 17 22:29:26.253646 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.253561    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0bca3-kube-rbac-proxy-sar-config\""
Sep 17 22:29:26.260844 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.260822    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"]
Sep 17 22:29:26.350469 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.350437    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.350627 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.350477    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdg5q\" (UniqueName: \"kubernetes.io/projected/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-api-access-bdg5q\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.350627 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.350513    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/843a0cfe-31af-49e5-a4f4-250e61e8a613-proxy-tls\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.451121 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.451090    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.451121 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.451124    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdg5q\" (UniqueName: \"kubernetes.io/projected/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-api-access-bdg5q\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.451397 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.451149    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/843a0cfe-31af-49e5-a4f4-250e61e8a613-proxy-tls\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.451810 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.451784    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.453653 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.453636    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/843a0cfe-31af-49e5-a4f4-250e61e8a613-proxy-tls\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.458994 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.458970    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdg5q\" (UniqueName: \"kubernetes.io/projected/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-api-access-bdg5q\") pod \"error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") " pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.562942 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.562870    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:26.830308 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:26.830283    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"]
Sep 17 22:29:26.832393 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:29:26.832357    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843a0cfe_31af_49e5_a4f4_250e61e8a613.slice/crio-1855652fe614ba8780fd183a04d115a9766cd9ee541732023a4f95ae80ece531 WatchSource:0}: Error finding container 1855652fe614ba8780fd183a04d115a9766cd9ee541732023a4f95ae80ece531: Status 404 returned error can't find the container with id 1855652fe614ba8780fd183a04d115a9766cd9ee541732023a4f95ae80ece531
Sep 17 22:29:27.238336 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:27.238296    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" event={"ID":"843a0cfe-31af-49e5-a4f4-250e61e8a613","Type":"ContainerStarted","Data":"d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c"}
Sep 17 22:29:27.238747 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:27.238362    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:27.238747 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:27.238393    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" event={"ID":"843a0cfe-31af-49e5-a4f4-250e61e8a613","Type":"ContainerStarted","Data":"c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac"}
Sep 17 22:29:27.238747 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:27.238418    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" event={"ID":"843a0cfe-31af-49e5-a4f4-250e61e8a613","Type":"ContainerStarted","Data":"1855652fe614ba8780fd183a04d115a9766cd9ee541732023a4f95ae80ece531"}
Sep 17 22:29:27.238747 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:27.238445    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:27.239985 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:27.239955    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 22:29:27.255252 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:27.255204    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podStartSLOduration=1.255187069 podStartE2EDuration="1.255187069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:29:27.25337848 +0000 UTC m=+1898.067753857" watchObservedRunningTime="2026-09-17 22:29:27.255187069 +0000 UTC m=+1898.069562446"
Sep 17 22:29:28.200337 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:28.200300    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:28.242721 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:28.242682    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 22:29:29.077426 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.077386    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:29.249414 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.249372    2561 generic.go:356] "Generic (PLEG): container finished" podID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerID="5139d6007481ebb66d0ea2fdb7743c3417ab9e5cd42a3a09031d536b6c332b55" exitCode=0
Sep 17 22:29:29.249740 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.249452    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" event={"ID":"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed","Type":"ContainerDied","Data":"5139d6007481ebb66d0ea2fdb7743c3417ab9e5cd42a3a09031d536b6c332b55"}
Sep 17 22:29:29.320341 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.320319    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:29:29.373462 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.373388    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-rbac-proxy-sar-config\") pod \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") "
Sep 17 22:29:29.373462 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.373432    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtq6f\" (UniqueName: \"kubernetes.io/projected/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-api-access-wtq6f\") pod \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") "
Sep 17 22:29:29.373671 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.373465    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls\") pod \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\" (UID: \"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed\") "
Sep 17 22:29:29.373782 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.373757    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" (UID: "a354cd7e-013f-4e42-9120-6e9f9f3dc4ed"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:29:29.375560 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.375533    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" (UID: "a354cd7e-013f-4e42-9120-6e9f9f3dc4ed"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:29:29.375659 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.375614    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-api-access-wtq6f" (OuterVolumeSpecName: "kube-api-access-wtq6f") pod "a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" (UID: "a354cd7e-013f-4e42-9120-6e9f9f3dc4ed"). InnerVolumeSpecName "kube-api-access-wtq6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:29:29.474705 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.474672    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:29.474705 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.474698    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtq6f\" (UniqueName: \"kubernetes.io/projected/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-kube-api-access-wtq6f\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:29.474705 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:29.474707    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:30.187470 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:30.187434    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 22:29:30.255590 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:30.255557    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"
Sep 17 22:29:30.255590 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:30.255571    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6" event={"ID":"a354cd7e-013f-4e42-9120-6e9f9f3dc4ed","Type":"ContainerDied","Data":"b7e3ff591e5fb64e24832ce669b50093d586992ed3313301690953d51cba74ce"}
Sep 17 22:29:30.256075 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:30.255649    2561 scope.go:117] "RemoveContainer" containerID="b36dc75e2b6e99ea1b74aae92eaba6a2b7ce8a99a2b492c70d38340a03c7a6dc"
Sep 17 22:29:30.264205 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:30.264187    2561 scope.go:117] "RemoveContainer" containerID="5139d6007481ebb66d0ea2fdb7743c3417ab9e5cd42a3a09031d536b6c332b55"
Sep 17 22:29:30.275268 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:30.275228    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"]
Sep 17 22:29:30.281135 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:30.281113    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2de1-predictor-7ff9c75c75-tmnf6"]
Sep 17 22:29:31.133112 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:31.133082    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" path="/var/lib/kubelet/pods/a354cd7e-013f-4e42-9120-6e9f9f3dc4ed/volumes"
Sep 17 22:29:33.200625 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:33.200585    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:33.246977 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:33.246953    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:29:33.247405 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:33.247376    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 22:29:34.078690 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:34.078652    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:38.200083 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:38.200046    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:38.200539 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:38.200143    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:39.077124 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:39.077082    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:40.188019 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:40.187978    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 22:29:41.917615 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:41.917592    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:29:41.980233 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:41.980207    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11cd1f95-1116-4cb7-9b48-0be92dc38471-openshift-service-ca-bundle\") pod \"11cd1f95-1116-4cb7-9b48-0be92dc38471\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") "
Sep 17 22:29:41.980375 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:41.980280    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11cd1f95-1116-4cb7-9b48-0be92dc38471-proxy-tls\") pod \"11cd1f95-1116-4cb7-9b48-0be92dc38471\" (UID: \"11cd1f95-1116-4cb7-9b48-0be92dc38471\") "
Sep 17 22:29:41.980613 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:41.980593    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11cd1f95-1116-4cb7-9b48-0be92dc38471-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "11cd1f95-1116-4cb7-9b48-0be92dc38471" (UID: "11cd1f95-1116-4cb7-9b48-0be92dc38471"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:29:41.982348 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:41.982323    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11cd1f95-1116-4cb7-9b48-0be92dc38471-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "11cd1f95-1116-4cb7-9b48-0be92dc38471" (UID: "11cd1f95-1116-4cb7-9b48-0be92dc38471"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:29:42.081518 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.081431    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11cd1f95-1116-4cb7-9b48-0be92dc38471-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:42.081518 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.081458    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11cd1f95-1116-4cb7-9b48-0be92dc38471-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:42.310055 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.310023    2561 generic.go:356] "Generic (PLEG): container finished" podID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerID="9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8" exitCode=0
Sep 17 22:29:42.310223 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.310071    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"
Sep 17 22:29:42.310223 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.310080    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" event={"ID":"11cd1f95-1116-4cb7-9b48-0be92dc38471","Type":"ContainerDied","Data":"9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8"}
Sep 17 22:29:42.310223 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.310125    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d" event={"ID":"11cd1f95-1116-4cb7-9b48-0be92dc38471","Type":"ContainerDied","Data":"8666d4a28df91e84820dc4aaeb1b5bfb41a7588c0cd8aa12ccada3b8db5fd7c5"}
Sep 17 22:29:42.310223 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.310148    2561 scope.go:117] "RemoveContainer" containerID="9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8"
Sep 17 22:29:42.319408 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.319388    2561 scope.go:117] "RemoveContainer" containerID="9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8"
Sep 17 22:29:42.319677 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:29:42.319658    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8\": container with ID starting with 9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8 not found: ID does not exist" containerID="9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8"
Sep 17 22:29:42.319725 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.319688    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8"} err="failed to get container status \"9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8\": rpc error: code = NotFound desc = could not find container \"9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8\": container with ID starting with 9477ac4c5dc4e485858d6a61860d8f87552c87ecb176d95cbc7ee5f2416aa8b8 not found: ID does not exist"
Sep 17 22:29:42.330054 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.330029    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"]
Sep 17 22:29:42.333484 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:42.333435    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-6bcee-76d5754-t957d"]
Sep 17 22:29:43.132353 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:43.132313    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" path="/var/lib/kubelet/pods/11cd1f95-1116-4cb7-9b48-0be92dc38471/volumes"
Sep 17 22:29:43.200689 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:43.200647    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:43.247863 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:43.247828    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 22:29:48.200452 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:48.200411    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:50.187558 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:50.187518    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 22:29:53.200954 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:53.200909    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:29:53.247498 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:53.247458    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 22:29:56.370280 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.370178    2561 generic.go:356] "Generic (PLEG): container finished" podID="d09d593d-d974-4fb1-92c7-688a9501197e" containerID="e0b0bc96ab423e2ff105d95bb183f2a29d299f706e3bde40909a68b9e7e6b10e" exitCode=0
Sep 17 22:29:56.370280 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.370269    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" event={"ID":"d09d593d-d974-4fb1-92c7-688a9501197e","Type":"ContainerDied","Data":"e0b0bc96ab423e2ff105d95bb183f2a29d299f706e3bde40909a68b9e7e6b10e"}
Sep 17 22:29:56.598845 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.598824    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:56.694973 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.694948    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d09d593d-d974-4fb1-92c7-688a9501197e-openshift-service-ca-bundle\") pod \"d09d593d-d974-4fb1-92c7-688a9501197e\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") "
Sep 17 22:29:56.695140 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.694981    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d593d-d974-4fb1-92c7-688a9501197e-proxy-tls\") pod \"d09d593d-d974-4fb1-92c7-688a9501197e\" (UID: \"d09d593d-d974-4fb1-92c7-688a9501197e\") "
Sep 17 22:29:56.695314 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.695290    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d09d593d-d974-4fb1-92c7-688a9501197e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d09d593d-d974-4fb1-92c7-688a9501197e" (UID: "d09d593d-d974-4fb1-92c7-688a9501197e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:29:56.697029 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.697007    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09d593d-d974-4fb1-92c7-688a9501197e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d09d593d-d974-4fb1-92c7-688a9501197e" (UID: "d09d593d-d974-4fb1-92c7-688a9501197e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:29:56.795479 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.795444    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d09d593d-d974-4fb1-92c7-688a9501197e-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:56.795479 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:56.795473    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d09d593d-d974-4fb1-92c7-688a9501197e-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:29:57.375928 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:57.375902    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"
Sep 17 22:29:57.375928 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:57.375912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95" event={"ID":"d09d593d-d974-4fb1-92c7-688a9501197e","Type":"ContainerDied","Data":"44044955329d912b7483a02327476628f2058647bd22f7ff82db004f098fd354"}
Sep 17 22:29:57.376421 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:57.375977    2561 scope.go:117] "RemoveContainer" containerID="e0b0bc96ab423e2ff105d95bb183f2a29d299f706e3bde40909a68b9e7e6b10e"
Sep 17 22:29:57.390175 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:57.390152    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"]
Sep 17 22:29:57.393662 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:57.393609    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a2de1-748b4b9f7d-wrx95"]
Sep 17 22:29:59.133221 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:29:59.133182    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" path="/var/lib/kubelet/pods/d09d593d-d974-4fb1-92c7-688a9501197e/volumes"
Sep 17 22:30:00.188140 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:00.188109    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:30:03.247382 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:03.247343    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 22:30:11.994550 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.994504    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"]
Sep 17 22:30:11.995350 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995329    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container"
Sep 17 22:30:11.995438 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995362    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container"
Sep 17 22:30:11.995438 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995379    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee"
Sep 17 22:30:11.995438 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995389    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee"
Sep 17 22:30:11.995438 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995420    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kube-rbac-proxy"
Sep 17 22:30:11.995438 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995429    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kube-rbac-proxy"
Sep 17 22:30:11.995702 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995453    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1"
Sep 17 22:30:11.995702 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995462    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1"
Sep 17 22:30:11.995702 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995605    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cd1f95-1116-4cb7-9b48-0be92dc38471" containerName="sequence-graph-6bcee"
Sep 17 22:30:11.995702 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995626    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kube-rbac-proxy"
Sep 17 22:30:11.995702 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995635    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d09d593d-d974-4fb1-92c7-688a9501197e" containerName="splitter-graph-a2de1"
Sep 17 22:30:11.995702 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:11.995650    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a354cd7e-013f-4e42-9120-6e9f9f3dc4ed" containerName="kserve-container"
Sep 17 22:30:12.003091 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.003066    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.004179 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.004152    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"]
Sep 17 22:30:12.005559 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.005515    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-94ae3-kube-rbac-proxy-sar-config\""
Sep 17 22:30:12.005681 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.005613    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-94ae3-serving-cert\""
Sep 17 22:30:12.015699 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.015665    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44dea073-a581-4edd-8293-620a4c6034c9-openshift-service-ca-bundle\") pod \"switch-graph-94ae3-7b768cddf-5nnwb\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") " pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.015807 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.015748    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44dea073-a581-4edd-8293-620a4c6034c9-proxy-tls\") pod \"switch-graph-94ae3-7b768cddf-5nnwb\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") " pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.116286 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.116266    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44dea073-a581-4edd-8293-620a4c6034c9-openshift-service-ca-bundle\") pod \"switch-graph-94ae3-7b768cddf-5nnwb\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") " pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.116391 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.116317    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44dea073-a581-4edd-8293-620a4c6034c9-proxy-tls\") pod \"switch-graph-94ae3-7b768cddf-5nnwb\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") " pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.116827 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.116811    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44dea073-a581-4edd-8293-620a4c6034c9-openshift-service-ca-bundle\") pod \"switch-graph-94ae3-7b768cddf-5nnwb\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") " pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.118672 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.118652    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44dea073-a581-4edd-8293-620a4c6034c9-proxy-tls\") pod \"switch-graph-94ae3-7b768cddf-5nnwb\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") " pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.313934 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.313852    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:12.581074 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:12.581049    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"]
Sep 17 22:30:12.583650 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:30:12.583612    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44dea073_a581_4edd_8293_620a4c6034c9.slice/crio-63cc66c083dc8259ea8d96409d46a43001fb53bf60f87f6e1411017559ab0527 WatchSource:0}: Error finding container 63cc66c083dc8259ea8d96409d46a43001fb53bf60f87f6e1411017559ab0527: Status 404 returned error can't find the container with id 63cc66c083dc8259ea8d96409d46a43001fb53bf60f87f6e1411017559ab0527
Sep 17 22:30:13.249095 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:13.249064    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:30:13.444075 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:13.444038    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" event={"ID":"44dea073-a581-4edd-8293-620a4c6034c9","Type":"ContainerStarted","Data":"5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb"}
Sep 17 22:30:13.444292 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:13.444088    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" event={"ID":"44dea073-a581-4edd-8293-620a4c6034c9","Type":"ContainerStarted","Data":"63cc66c083dc8259ea8d96409d46a43001fb53bf60f87f6e1411017559ab0527"}
Sep 17 22:30:13.444292 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:13.444144    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:13.459645 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:13.459604    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podStartSLOduration=2.459590442 podStartE2EDuration="2.459590442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:30:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:30:13.457436727 +0000 UTC m=+1944.271812104" watchObservedRunningTime="2026-09-17 22:30:13.459590442 +0000 UTC m=+1944.273965820"
Sep 17 22:30:19.453832 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:19.453803    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:30:26.176549 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.176514    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"]
Sep 17 22:30:26.180061 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.180039    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.182284 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.182241    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-0bca3-serving-cert\""
Sep 17 22:30:26.182395 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.182264    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-0bca3-kube-rbac-proxy-sar-config\""
Sep 17 22:30:26.190439 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.190416    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"]
Sep 17 22:30:26.219681 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.219651    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6393a878-92d2-4c1f-8236-560a53d74c54-openshift-service-ca-bundle\") pod \"splitter-graph-0bca3-868ddc5d59-8hxx9\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") " pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.219794 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.219691    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6393a878-92d2-4c1f-8236-560a53d74c54-proxy-tls\") pod \"splitter-graph-0bca3-868ddc5d59-8hxx9\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") " pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.320059 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.320024    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6393a878-92d2-4c1f-8236-560a53d74c54-openshift-service-ca-bundle\") pod \"splitter-graph-0bca3-868ddc5d59-8hxx9\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") " pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.320059 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.320062    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6393a878-92d2-4c1f-8236-560a53d74c54-proxy-tls\") pod \"splitter-graph-0bca3-868ddc5d59-8hxx9\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") " pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.320676 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.320657    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6393a878-92d2-4c1f-8236-560a53d74c54-openshift-service-ca-bundle\") pod \"splitter-graph-0bca3-868ddc5d59-8hxx9\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") " pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.322380 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.322359    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6393a878-92d2-4c1f-8236-560a53d74c54-proxy-tls\") pod \"splitter-graph-0bca3-868ddc5d59-8hxx9\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") " pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.491238 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.491151    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:26.759895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:26.759867    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"]
Sep 17 22:30:26.762410 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:30:26.762373    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6393a878_92d2_4c1f_8236_560a53d74c54.slice/crio-ab66b6a580659273be46743441b33b64db6118234302671c6cf4af5df7fb565b WatchSource:0}: Error finding container ab66b6a580659273be46743441b33b64db6118234302671c6cf4af5df7fb565b: Status 404 returned error can't find the container with id ab66b6a580659273be46743441b33b64db6118234302671c6cf4af5df7fb565b
Sep 17 22:30:27.506560 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:27.506524    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" event={"ID":"6393a878-92d2-4c1f-8236-560a53d74c54","Type":"ContainerStarted","Data":"5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0"}
Sep 17 22:30:27.507036 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:27.506575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" event={"ID":"6393a878-92d2-4c1f-8236-560a53d74c54","Type":"ContainerStarted","Data":"ab66b6a580659273be46743441b33b64db6118234302671c6cf4af5df7fb565b"}
Sep 17 22:30:27.507036 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:27.506665    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:30:27.521181 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:27.521139    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podStartSLOduration=1.521125755 podStartE2EDuration="1.521125755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 22:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:30:27.520600889 +0000 UTC m=+1958.334976262" watchObservedRunningTime="2026-09-17 22:30:27.521125755 +0000 UTC m=+1958.335501131"
Sep 17 22:30:33.517152 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:30:33.517115    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:32:51.223285 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:32:51.223240    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:32:51.229099 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:32:51.229078    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:37:51.248278 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:37:51.248153    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:37:51.255239 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:37:51.255219    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:38:40.741693 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:40.741655    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"]
Sep 17 22:38:40.742326 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:40.741909    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3" containerID="cri-o://5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0" gracePeriod=30
Sep 17 22:38:40.867424 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:40.867393    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"]
Sep 17 22:38:40.867680 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:40.867630    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" containerID="cri-o://c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac" gracePeriod=30
Sep 17 22:38:40.867831 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:40.867666    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kube-rbac-proxy" containerID="cri-o://d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c" gracePeriod=30
Sep 17 22:38:41.623807 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:41.623765    2561 generic.go:356] "Generic (PLEG): container finished" podID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerID="d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c" exitCode=2
Sep 17 22:38:41.623990 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:41.623846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" event={"ID":"843a0cfe-31af-49e5-a4f4-250e61e8a613","Type":"ContainerDied","Data":"d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c"}
Sep 17 22:38:43.243256 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:43.243216    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.48:8643/healthz\": dial tcp 10.133.0.48:8643: connect: connection refused"
Sep 17 22:38:43.247521 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:43.247489    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 22:38:43.514786 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:43.514690    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:38:44.005219 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.005194    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:38:44.069419 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.069339    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-rbac-proxy-sar-config\") pod \"843a0cfe-31af-49e5-a4f4-250e61e8a613\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") "
Sep 17 22:38:44.069419 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.069374    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdg5q\" (UniqueName: \"kubernetes.io/projected/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-api-access-bdg5q\") pod \"843a0cfe-31af-49e5-a4f4-250e61e8a613\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") "
Sep 17 22:38:44.069621 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.069419    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/843a0cfe-31af-49e5-a4f4-250e61e8a613-proxy-tls\") pod \"843a0cfe-31af-49e5-a4f4-250e61e8a613\" (UID: \"843a0cfe-31af-49e5-a4f4-250e61e8a613\") "
Sep 17 22:38:44.069752 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.069729    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "843a0cfe-31af-49e5-a4f4-250e61e8a613" (UID: "843a0cfe-31af-49e5-a4f4-250e61e8a613"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:38:44.071549 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.071520    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843a0cfe-31af-49e5-a4f4-250e61e8a613-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "843a0cfe-31af-49e5-a4f4-250e61e8a613" (UID: "843a0cfe-31af-49e5-a4f4-250e61e8a613"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:38:44.071678 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.071553    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-api-access-bdg5q" (OuterVolumeSpecName: "kube-api-access-bdg5q") pod "843a0cfe-31af-49e5-a4f4-250e61e8a613" (UID: "843a0cfe-31af-49e5-a4f4-250e61e8a613"). InnerVolumeSpecName "kube-api-access-bdg5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:38:44.170145 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.170114    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:38:44.170145 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.170139    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdg5q\" (UniqueName: \"kubernetes.io/projected/843a0cfe-31af-49e5-a4f4-250e61e8a613-kube-api-access-bdg5q\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:38:44.170145 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.170149    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/843a0cfe-31af-49e5-a4f4-250e61e8a613-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:38:44.639633 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.639599    2561 generic.go:356] "Generic (PLEG): container finished" podID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerID="c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac" exitCode=0
Sep 17 22:38:44.640044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.639652    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"
Sep 17 22:38:44.640044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.639661    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" event={"ID":"843a0cfe-31af-49e5-a4f4-250e61e8a613","Type":"ContainerDied","Data":"c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac"}
Sep 17 22:38:44.640044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.639710    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr" event={"ID":"843a0cfe-31af-49e5-a4f4-250e61e8a613","Type":"ContainerDied","Data":"1855652fe614ba8780fd183a04d115a9766cd9ee541732023a4f95ae80ece531"}
Sep 17 22:38:44.640044 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.639735    2561 scope.go:117] "RemoveContainer" containerID="d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c"
Sep 17 22:38:44.648224 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.648207    2561 scope.go:117] "RemoveContainer" containerID="c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac"
Sep 17 22:38:44.655636 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.655618    2561 scope.go:117] "RemoveContainer" containerID="d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c"
Sep 17 22:38:44.655899 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:38:44.655879    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c\": container with ID starting with d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c not found: ID does not exist" containerID="d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c"
Sep 17 22:38:44.655944 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.655908    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c"} err="failed to get container status \"d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c\": rpc error: code = NotFound desc = could not find container \"d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c\": container with ID starting with d8ccff2182adca7b41dd4eb24b8b25e68ce3a0ba51041a6e8b1b7cade1c4813c not found: ID does not exist"
Sep 17 22:38:44.655944 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.655925    2561 scope.go:117] "RemoveContainer" containerID="c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac"
Sep 17 22:38:44.656175 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:38:44.656157    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac\": container with ID starting with c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac not found: ID does not exist" containerID="c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac"
Sep 17 22:38:44.656230 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.656180    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac"} err="failed to get container status \"c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac\": rpc error: code = NotFound desc = could not find container \"c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac\": container with ID starting with c2a5b9d51f047c12b9f1cb63210b766b50089a5e809f08b480e75ea1ef936bac not found: ID does not exist"
Sep 17 22:38:44.659537 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.659515    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"]
Sep 17 22:38:44.662498 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:44.662478    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0bca3-predictor-7689fbcd47-zjvdr"]
Sep 17 22:38:45.133544 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:45.133510    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" path="/var/lib/kubelet/pods/843a0cfe-31af-49e5-a4f4-250e61e8a613/volumes"
Sep 17 22:38:48.515536 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:48.515492    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:38:53.514603 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:53.514561    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:38:53.515001 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:53.514658    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:38:58.515027 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:38:58.514978    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:39:03.515735 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:03.515695    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:39:08.514836 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:08.514798    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:39:10.888482 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:10.888455    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:39:10.986833 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:10.986802    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6393a878-92d2-4c1f-8236-560a53d74c54-proxy-tls\") pod \"6393a878-92d2-4c1f-8236-560a53d74c54\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") "
Sep 17 22:39:10.987000 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:10.986852    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6393a878-92d2-4c1f-8236-560a53d74c54-openshift-service-ca-bundle\") pod \"6393a878-92d2-4c1f-8236-560a53d74c54\" (UID: \"6393a878-92d2-4c1f-8236-560a53d74c54\") "
Sep 17 22:39:10.987228 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:10.987206    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6393a878-92d2-4c1f-8236-560a53d74c54-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "6393a878-92d2-4c1f-8236-560a53d74c54" (UID: "6393a878-92d2-4c1f-8236-560a53d74c54"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:39:10.988838 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:10.988817    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6393a878-92d2-4c1f-8236-560a53d74c54-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6393a878-92d2-4c1f-8236-560a53d74c54" (UID: "6393a878-92d2-4c1f-8236-560a53d74c54"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:39:11.088270 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.088192    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6393a878-92d2-4c1f-8236-560a53d74c54-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:39:11.088270 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.088216    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6393a878-92d2-4c1f-8236-560a53d74c54-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:39:11.760381 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.760350    2561 generic.go:356] "Generic (PLEG): container finished" podID="6393a878-92d2-4c1f-8236-560a53d74c54" containerID="5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0" exitCode=0
Sep 17 22:39:11.760601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.760393    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"
Sep 17 22:39:11.760601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.760440    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" event={"ID":"6393a878-92d2-4c1f-8236-560a53d74c54","Type":"ContainerDied","Data":"5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0"}
Sep 17 22:39:11.760601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.760499    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9" event={"ID":"6393a878-92d2-4c1f-8236-560a53d74c54","Type":"ContainerDied","Data":"ab66b6a580659273be46743441b33b64db6118234302671c6cf4af5df7fb565b"}
Sep 17 22:39:11.760601 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.760535    2561 scope.go:117] "RemoveContainer" containerID="5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0"
Sep 17 22:39:11.768344 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.768327    2561 scope.go:117] "RemoveContainer" containerID="5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0"
Sep 17 22:39:11.768616 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:39:11.768587    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0\": container with ID starting with 5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0 not found: ID does not exist" containerID="5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0"
Sep 17 22:39:11.768714 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.768616    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0"} err="failed to get container status \"5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0\": rpc error: code = NotFound desc = could not find container \"5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0\": container with ID starting with 5061b886cf7bb980e639c317c8bff9b8aa7bf1f3eb9314ee56ace34898313eb0 not found: ID does not exist"
Sep 17 22:39:11.775339 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.775317    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"]
Sep 17 22:39:11.778367 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:11.778348    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0bca3-868ddc5d59-8hxx9"]
Sep 17 22:39:13.132135 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:39:13.132103    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" path="/var/lib/kubelet/pods/6393a878-92d2-4c1f-8236-560a53d74c54/volumes"
Sep 17 22:42:51.271861 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:42:51.271752    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:42:51.279020 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:42:51.279000    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:46:31.297002 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:31.296963    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"]
Sep 17 22:46:31.297643 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:31.297170    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3" containerID="cri-o://5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb" gracePeriod=30
Sep 17 22:46:31.433082 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:31.433047    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"]
Sep 17 22:46:31.433359 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:31.433329    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container" containerID="cri-o://538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310" gracePeriod=30
Sep 17 22:46:31.433466 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:31.433366    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kube-rbac-proxy" containerID="cri-o://8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18" gracePeriod=30
Sep 17 22:46:31.597933 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:31.597850    2561 generic.go:356] "Generic (PLEG): container finished" podID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerID="8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18" exitCode=2
Sep 17 22:46:31.597933 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:31.597898    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" event={"ID":"b182630b-eec4-4342-98e9-a5f0c3b5b478","Type":"ContainerDied","Data":"8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18"}
Sep 17 22:46:34.453398 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.453360    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:46:34.468827 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.468802    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:46:34.519568 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.519536    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjnpw\" (UniqueName: \"kubernetes.io/projected/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-api-access-xjnpw\") pod \"b182630b-eec4-4342-98e9-a5f0c3b5b478\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") "
Sep 17 22:46:34.519743 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.519578    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b182630b-eec4-4342-98e9-a5f0c3b5b478-proxy-tls\") pod \"b182630b-eec4-4342-98e9-a5f0c3b5b478\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") "
Sep 17 22:46:34.519743 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.519621    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-rbac-proxy-sar-config\") pod \"b182630b-eec4-4342-98e9-a5f0c3b5b478\" (UID: \"b182630b-eec4-4342-98e9-a5f0c3b5b478\") "
Sep 17 22:46:34.519990 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.519968    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b182630b-eec4-4342-98e9-a5f0c3b5b478" (UID: "b182630b-eec4-4342-98e9-a5f0c3b5b478"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:46:34.521617 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.521594    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b182630b-eec4-4342-98e9-a5f0c3b5b478-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b182630b-eec4-4342-98e9-a5f0c3b5b478" (UID: "b182630b-eec4-4342-98e9-a5f0c3b5b478"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:46:34.521684 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.521643    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-api-access-xjnpw" (OuterVolumeSpecName: "kube-api-access-xjnpw") pod "b182630b-eec4-4342-98e9-a5f0c3b5b478" (UID: "b182630b-eec4-4342-98e9-a5f0c3b5b478"). InnerVolumeSpecName "kube-api-access-xjnpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 22:46:34.610782 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.610699    2561 generic.go:356] "Generic (PLEG): container finished" podID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerID="538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310" exitCode=0
Sep 17 22:46:34.610782 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.610742    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"
Sep 17 22:46:34.610963 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.610744    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" event={"ID":"b182630b-eec4-4342-98e9-a5f0c3b5b478","Type":"ContainerDied","Data":"538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310"}
Sep 17 22:46:34.610963 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.610882    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r" event={"ID":"b182630b-eec4-4342-98e9-a5f0c3b5b478","Type":"ContainerDied","Data":"e7870973f15121609d884d661ce4c41db004de76bb3b5c046e28a64de5332f01"}
Sep 17 22:46:34.610963 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.610915    2561 scope.go:117] "RemoveContainer" containerID="8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18"
Sep 17 22:46:34.619666 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.619646    2561 scope.go:117] "RemoveContainer" containerID="538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310"
Sep 17 22:46:34.620186 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.620168    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjnpw\" (UniqueName: \"kubernetes.io/projected/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-api-access-xjnpw\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:46:34.620302 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.620191    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b182630b-eec4-4342-98e9-a5f0c3b5b478-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:46:34.620302 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.620206    2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b182630b-eec4-4342-98e9-a5f0c3b5b478-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:46:34.626636 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.626620    2561 scope.go:117] "RemoveContainer" containerID="8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18"
Sep 17 22:46:34.626891 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:46:34.626874    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18\": container with ID starting with 8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18 not found: ID does not exist" containerID="8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18"
Sep 17 22:46:34.626939 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.626899    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18"} err="failed to get container status \"8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18\": rpc error: code = NotFound desc = could not find container \"8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18\": container with ID starting with 8bb6cd63a61a5c60866ecc6a81756f3e205528f9c076a34d796c61c1da19ad18 not found: ID does not exist"
Sep 17 22:46:34.626939 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.626915    2561 scope.go:117] "RemoveContainer" containerID="538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310"
Sep 17 22:46:34.627150 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:46:34.627135    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310\": container with ID starting with 538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310 not found: ID does not exist" containerID="538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310"
Sep 17 22:46:34.627199 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.627154    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310"} err="failed to get container status \"538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310\": rpc error: code = NotFound desc = could not find container \"538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310\": container with ID starting with 538c5d316fc88539b945b8f5a1e249be9f1851d811b95a9aae2a72313692d310 not found: ID does not exist"
Sep 17 22:46:34.631550 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.631528    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"]
Sep 17 22:46:34.634844 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:34.634826    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94ae3-predictor-7dd5ccf689-j665r"]
Sep 17 22:46:35.132911 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:35.132880    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" path="/var/lib/kubelet/pods/b182630b-eec4-4342-98e9-a5f0c3b5b478/volumes"
Sep 17 22:46:39.453116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:39.453071    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:46:44.453322 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:44.453279    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:46:44.453684 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:44.453383    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:46:47.364642 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:47.364605    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:48.118137 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:48.118111    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:48.863451 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:48.863415    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:49.453265 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:49.453211    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:46:49.589366 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:49.589337    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:50.319694 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:50.319663    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:51.049450 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:51.049420    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:51.764355 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:51.764273    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:52.480577 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:52.480549    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:53.234125 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:53.234098    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:53.956571 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:53.956539    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:54.453119 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:54.453076    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:46:54.668624 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:54.668591    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:55.382554 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:55.382513    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:56.094138 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:56.094109    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:56.819469 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:56.819434    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:57.541827 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:57.541800    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-94ae3-7b768cddf-5nnwb_44dea073-a581-4edd-8293-620a4c6034c9/switch-graph-94ae3/0.log"
Sep 17 22:46:59.453527 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:46:59.453484    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 22:47:01.439886 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.439861    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:47:01.540116 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.540085    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44dea073-a581-4edd-8293-620a4c6034c9-proxy-tls\") pod \"44dea073-a581-4edd-8293-620a4c6034c9\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") "
Sep 17 22:47:01.540325 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.540138    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44dea073-a581-4edd-8293-620a4c6034c9-openshift-service-ca-bundle\") pod \"44dea073-a581-4edd-8293-620a4c6034c9\" (UID: \"44dea073-a581-4edd-8293-620a4c6034c9\") "
Sep 17 22:47:01.540484 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.540461    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44dea073-a581-4edd-8293-620a4c6034c9-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "44dea073-a581-4edd-8293-620a4c6034c9" (UID: "44dea073-a581-4edd-8293-620a4c6034c9"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 22:47:01.542189 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.542166    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44dea073-a581-4edd-8293-620a4c6034c9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "44dea073-a581-4edd-8293-620a4c6034c9" (UID: "44dea073-a581-4edd-8293-620a4c6034c9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 22:47:01.641622 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.641528    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44dea073-a581-4edd-8293-620a4c6034c9-openshift-service-ca-bundle\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:47:01.641622 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.641568    2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44dea073-a581-4edd-8293-620a4c6034c9-proxy-tls\") on node \"ip-10-0-131-69.ec2.internal\" DevicePath \"\""
Sep 17 22:47:01.727556 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.727522    2561 generic.go:356] "Generic (PLEG): container finished" podID="44dea073-a581-4edd-8293-620a4c6034c9" containerID="5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb" exitCode=0
Sep 17 22:47:01.727716 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.727560    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"
Sep 17 22:47:01.727716 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.727609    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" event={"ID":"44dea073-a581-4edd-8293-620a4c6034c9","Type":"ContainerDied","Data":"5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb"}
Sep 17 22:47:01.727716 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.727691    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb" event={"ID":"44dea073-a581-4edd-8293-620a4c6034c9","Type":"ContainerDied","Data":"63cc66c083dc8259ea8d96409d46a43001fb53bf60f87f6e1411017559ab0527"}
Sep 17 22:47:01.727716 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.727713    2561 scope.go:117] "RemoveContainer" containerID="5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb"
Sep 17 22:47:01.736061 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.736045    2561 scope.go:117] "RemoveContainer" containerID="5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb"
Sep 17 22:47:01.736374 ip-10-0-131-69 kubenswrapper[2561]: E0917 22:47:01.736352    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb\": container with ID starting with 5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb not found: ID does not exist" containerID="5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb"
Sep 17 22:47:01.736430 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.736383    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb"} err="failed to get container status \"5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb\": rpc error: code = NotFound desc = could not find container \"5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb\": container with ID starting with 5c5590db8b185d9cf58ec1363273b8abb9b84846866baf645f694b784cb544fb not found: ID does not exist"
Sep 17 22:47:01.747656 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.747613    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"]
Sep 17 22:47:01.749769 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:01.749745    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-94ae3-7b768cddf-5nnwb"]
Sep 17 22:47:03.133076 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:03.133043    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44dea073-a581-4edd-8293-620a4c6034c9" path="/var/lib/kubelet/pods/44dea073-a581-4edd-8293-620a4c6034c9/volumes"
Sep 17 22:47:06.089606 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:06.089513    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-djhj9_9811d2bc-c727-42e3-a514-02ca5a24be2f/global-pull-secret-syncer/0.log"
Sep 17 22:47:06.235404 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:06.235374    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tpvqh_dbd374c7-122a-4732-808f-9777edcf8fa9/konnectivity-agent/0.log"
Sep 17 22:47:06.275659 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:06.275636    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-69.ec2.internal_cb7e1d05bdbb0cfb0bbd514a8ce678b1/haproxy/0.log"
Sep 17 22:47:10.818187 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:10.818155    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-tbfxh_4d0171e6-885b-472b-bc98-38054e993d70/monitoring-plugin/0.log"
Sep 17 22:47:10.850800 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:10.850773    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9sjh_773ee54f-7190-4537-93b2-777967b0b70c/node-exporter/0.log"
Sep 17 22:47:10.873334 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:10.873309    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9sjh_773ee54f-7190-4537-93b2-777967b0b70c/kube-rbac-proxy/0.log"
Sep 17 22:47:10.896302 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:10.896285    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9sjh_773ee54f-7190-4537-93b2-777967b0b70c/init-textfile/0.log"
Sep 17 22:47:11.323434 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:11.323407    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-zrl4z_5c897a44-f7ce-48ec-8c6f-320fe94c7a2e/prometheus-operator-admission-webhook/0.log"
Sep 17 22:47:13.689401 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:13.689378    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5688df7978-9ctpx_c030a11f-4607-4e1f-9a28-c4ddf4c2bb44/console/0.log"
Sep 17 22:47:13.716845 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:13.716817    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-wb4nh_16d7f283-755e-49d9-944b-986fa7dec726/download-server/0.log"
Sep 17 22:47:14.404284 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404240    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"]
Sep 17 22:47:14.404598 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404584    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kube-rbac-proxy"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404600    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kube-rbac-proxy"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404612    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kube-rbac-proxy"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404618    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kube-rbac-proxy"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404630    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404636    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404644    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404649    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3"
Sep 17 22:47:14.404654 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404656    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404662    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404673    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404678    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404720    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kserve-container"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404728    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6393a878-92d2-4c1f-8236-560a53d74c54" containerName="splitter-graph-0bca3"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404734    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="44dea073-a581-4edd-8293-620a4c6034c9" containerName="switch-graph-94ae3"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404740    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="843a0cfe-31af-49e5-a4f4-250e61e8a613" containerName="kube-rbac-proxy"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404745    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kserve-container"
Sep 17 22:47:14.404895 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.404750    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b182630b-eec4-4342-98e9-a5f0c3b5b478" containerName="kube-rbac-proxy"
Sep 17 22:47:14.408863 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.408843    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.411127 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.411105    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b796f\"/\"kube-root-ca.crt\""
Sep 17 22:47:14.411989 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.411970    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b796f\"/\"openshift-service-ca.crt\""
Sep 17 22:47:14.412091 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.411971    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b796f\"/\"default-dockercfg-qblds\""
Sep 17 22:47:14.416057 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.416039    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"]
Sep 17 22:47:14.430235 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.430213    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-lib-modules\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.430344 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.430239    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-podres\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.430344 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.430316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-proc\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.430435 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.430358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-sys\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.430435 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.430394    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfx87\" (UniqueName: \"kubernetes.io/projected/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-kube-api-access-xfx87\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531519 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531489    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-lib-modules\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531519 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531521    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-podres\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531749 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531547    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-proc\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531749 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531570    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-sys\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531749 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531609    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfx87\" (UniqueName: \"kubernetes.io/projected/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-kube-api-access-xfx87\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531749 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531650    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-proc\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531749 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531675    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-lib-modules\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531749 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531700    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-sys\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.531749 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.531675    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-podres\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.538814 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.538794    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfx87\" (UniqueName: \"kubernetes.io/projected/ec6b1d00-2e8d-46f8-99cc-225d4aa37a87-kube-api-access-xfx87\") pod \"perf-node-gather-daemonset-7b67s\" (UID: \"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87\") " pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.719006 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.718916    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:14.986824 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:14.986748    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"]
Sep 17 22:47:14.989729 ip-10-0-131-69 kubenswrapper[2561]: W0917 22:47:14.989690    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podec6b1d00_2e8d_46f8_99cc_225d4aa37a87.slice/crio-94b5bc08078a2fc651548f647572e130d47d426ae756bc9d6b4ae95b4464c3ce WatchSource:0}: Error finding container 94b5bc08078a2fc651548f647572e130d47d426ae756bc9d6b4ae95b4464c3ce: Status 404 returned error can't find the container with id 94b5bc08078a2fc651548f647572e130d47d426ae756bc9d6b4ae95b4464c3ce
Sep 17 22:47:15.037100 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.037081    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gbdmh_ddcb190d-cc99-43e6-b1ba-59a1b0450888/dns/0.log"
Sep 17 22:47:15.054133 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.054113    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gbdmh_ddcb190d-cc99-43e6-b1ba-59a1b0450888/kube-rbac-proxy/0.log"
Sep 17 22:47:15.112397 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.112374    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6swmf_e536b96a-1b2e-4b48-96d6-128570aca0cc/dns-node-resolver/0.log"
Sep 17 22:47:15.580143 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.580115    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5c75b475d-d8cmf_7dff0f76-948e-4e1a-bef1-f9013a4d8bc5/registry/0.log"
Sep 17 22:47:15.624705 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.624678    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-msktq_87535f94-4b13-4fb8-a307-eb2fdf5ec1d0/node-ca/0.log"
Sep 17 22:47:15.788777 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.788735    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s" event={"ID":"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87","Type":"ContainerStarted","Data":"de5f94d9c4ec5f4b3eac575beab08be8ff9f1885229b2f82c3d85406805ef26e"}
Sep 17 22:47:15.789145 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.788815    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:15.789145 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.788841    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s" event={"ID":"ec6b1d00-2e8d-46f8-99cc-225d4aa37a87","Type":"ContainerStarted","Data":"94b5bc08078a2fc651548f647572e130d47d426ae756bc9d6b4ae95b4464c3ce"}
Sep 17 22:47:15.804008 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:15.803954    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s" podStartSLOduration=1.803936855 podStartE2EDuration="1.803936855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 22:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 22:47:15.802395266 +0000 UTC m=+2966.616770643" watchObservedRunningTime="2026-09-17 22:47:15.803936855 +0000 UTC m=+2966.618312232"
Sep 17 22:47:16.667132 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:16.667103    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bw55n_e6a474d1-7b6b-4408-945d-0125b4f216ec/serve-healthcheck-canary/0.log"
Sep 17 22:47:17.186113 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:17.186086    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xnfbw_d1a1026a-0e11-4e8b-8e43-1f433fe41480/kube-rbac-proxy/0.log"
Sep 17 22:47:17.203005 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:17.202984    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xnfbw_d1a1026a-0e11-4e8b-8e43-1f433fe41480/exporter/0.log"
Sep 17 22:47:17.220803 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:17.220778    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xnfbw_d1a1026a-0e11-4e8b-8e43-1f433fe41480/extractor/0.log"
Sep 17 22:47:19.527898 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:19.527869    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-9rct5_648b64cb-e97e-4e27-a477-65a0177896b1/server/0.log"
Sep 17 22:47:19.608858 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:19.608812    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-cs95q_6e732cb9-e64e-475f-bd69-d78cb6ddcaa0/manager/0.log"
Sep 17 22:47:19.686475 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:19.686446    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-hxqdc_c037a4f8-7dbc-420f-a585-538ee81c5078/seaweedfs/0.log"
Sep 17 22:47:21.801632 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:21.801603    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b796f/perf-node-gather-daemonset-7b67s"
Sep 17 22:47:23.403179 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:23.403148    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-zzvgs_49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc/migrator/0.log"
Sep 17 22:47:23.421400 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:23.421345    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-zzvgs_49c5740f-9bf0-41cb-90f8-2b9aa9ee51fc/graceful-termination/0.log"
Sep 17 22:47:24.655594 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:24.655570    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5qc2s_f10417bd-9a32-43ea-be80-6a2b4a414203/kube-multus-additional-cni-plugins/0.log"
Sep 17 22:47:24.672461 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:24.672439    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5qc2s_f10417bd-9a32-43ea-be80-6a2b4a414203/egress-router-binary-copy/0.log"
Sep 17 22:47:24.689156 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:24.689135    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5qc2s_f10417bd-9a32-43ea-be80-6a2b4a414203/cni-plugins/0.log"
Sep 17 22:47:24.708090 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:24.708074    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5qc2s_f10417bd-9a32-43ea-be80-6a2b4a414203/bond-cni-plugin/0.log"
Sep 17 22:47:24.727847 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:24.727833    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5qc2s_f10417bd-9a32-43ea-be80-6a2b4a414203/routeoverride-cni/0.log"
Sep 17 22:47:24.745341 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:24.745322    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5qc2s_f10417bd-9a32-43ea-be80-6a2b4a414203/whereabouts-cni-bincopy/0.log"
Sep 17 22:47:24.763508 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:24.763487    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5qc2s_f10417bd-9a32-43ea-be80-6a2b4a414203/whereabouts-cni/0.log"
Sep 17 22:47:25.060638 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.060609    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bxzz8_776e2517-6042-4c45-b5ce-267e540962ad/kube-multus/0.log"
Sep 17 22:47:25.160569 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.160539    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xwm5p_2e8cccc3-163f-4c1e-af6e-6390d5a2a694/network-metrics-daemon/0.log"
Sep 17 22:47:25.179161 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.179102    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xwm5p_2e8cccc3-163f-4c1e-af6e-6390d5a2a694/kube-rbac-proxy/0.log"
Sep 17 22:47:25.912108 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.912083    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-controller/0.log"
Sep 17 22:47:25.931317 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.931295    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/1.log"
Sep 17 22:47:25.931581 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.931568    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovn-acl-logging/0.log"
Sep 17 22:47:25.946170 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.946153    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/kube-rbac-proxy-node/0.log"
Sep 17 22:47:25.964548 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.964528    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 22:47:25.982149 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.982131    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/northd/0.log"
Sep 17 22:47:25.998991 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:25.998973    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/nbdb/0.log"
Sep 17 22:47:26.016921 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:26.016902    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/sbdb/0.log"
Sep 17 22:47:26.051919 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:26.051899    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-24w8t_67602f97-3cbe-4453-aac5-114689901c4a/ovnkube-controller/0.log"
Sep 17 22:47:27.371136 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:27.371106    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-k7cfk_b3a32913-e0bd-40ff-b8c9-e80a8a7adc58/network-check-target-container/0.log"
Sep 17 22:47:28.239559 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:28.239530    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-cbzc2_aa8136e8-83ac-4e34-ae35-c6c11d62f18f/iptables-alerter/0.log"
Sep 17 22:47:28.868048 ip-10-0-131-69 kubenswrapper[2561]: I0917 22:47:28.868020    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-hx8mn_5fe371e6-2152-4ec7-8751-5b6d268b1e01/tuned/0.log"