Aug 04 18:00:39.202389 ip-10-0-136-239 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 18:00:39.569116 ip-10-0-136-239 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 18:00:39.569116 ip-10-0-136-239 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 18:00:39.569116 ip-10-0-136-239 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 18:00:39.569116 ip-10-0-136-239 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 18:00:39.569116 ip-10-0-136-239 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 18:00:39.570093 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.570012    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 18:00:39.574958 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574943    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:00:39.574958 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574957    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574961    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574966    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574969    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574972    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574975    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574978    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574981    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574984    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574986    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574990    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574992    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574995    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.574998    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575000    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575003    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575005    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575008    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575011    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575013    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:00:39.575020 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575016    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575018    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575021    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575024    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575027    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575030    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575033    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575036    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575039    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575042    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575044    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575047    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575049    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575052    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575054    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575057    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575059    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575062    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575064    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:00:39.575512 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575067    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575069    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575072    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575075    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575077    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575079    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575082    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575085    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575087    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575091    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575093    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575096    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575110    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575112    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575116    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575119    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575122    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575124    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575127    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575129    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:00:39.575970 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575132    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575135    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575137    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575140    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575143    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575145    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575148    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575150    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575153    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575156    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575158    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575161    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575163    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575166    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575168    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575171    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575174    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575176    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575178    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575181    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:00:39.576491 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575183    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575186    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575188    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575192    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575196    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575199    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575202    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575207    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575210    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575213    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575216    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575218    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575221    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575225    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575229    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575233    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575237    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575239    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:00:39.576980 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575243    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575912    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575919    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575922    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575926    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575929    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575932    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575935    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575938    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575940    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575943    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575946    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575948    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575951    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575954    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575958    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575961    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575964    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575967    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575970    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:00:39.577424 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575973    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575976    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575979    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575981    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575984    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575986    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575989    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575992    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575994    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575997    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.575999    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576002    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576005    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576007    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576009    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576012    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576015    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576017    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576020    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576022    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:00:39.577932 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576025    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576028    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576030    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576033    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576035    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576038    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576041    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576043    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576047    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576050    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576052    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576055    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576059    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576061    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576064    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576067    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576069    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576072    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576074    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576077    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:00:39.578443 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576080    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576082    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576085    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576087    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576090    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576093    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576095    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576111    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576114    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576116    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576119    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576122    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576128    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576131    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576133    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576136    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576138    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576141    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576145    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:00:39.578943 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576149    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576152    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576155    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576158    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576160    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576164    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576167    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576169    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576172    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576175    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576177    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576180    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576182    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576187    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576191    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576194    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576196    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576199    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576202    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:00:39.579428 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576205    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576208    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576580    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576585    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576588    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576592    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576595    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576598    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576600    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576603    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576606    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576608    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576613    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576616    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576620    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576623    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576626    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576629    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576632    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576634    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:00:39.579901 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576637    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576639    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576642    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576645    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576648    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576650    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576653    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576655    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576657    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576660    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576663    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576665    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576668    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576671    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576673    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576676    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576678    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576681    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576684    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576687    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:00:39.580425 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576689    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576692    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576694    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576697    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576699    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576702    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576705    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576707    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576710    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576712    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576715    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576717    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576720    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576723    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576725    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576728    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576730    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576733    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576736    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:00:39.580993 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576738    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576740    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576744    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576747    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576750    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576754    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576758    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576761    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576764    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576767    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576770    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576773    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576776    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576782    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576784    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576787    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576789    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576792    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576794    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:00:39.581511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576797    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576799    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576802    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576804    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576807    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576810    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576812    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576815    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576817    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576819    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576822    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576824    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576827    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576830    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576832    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576835    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576837    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576840    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576842    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576847    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:00:39.581998 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576849    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576852    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.576855    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576916    2562 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576923    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576928    2562 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576932    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576937    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576940    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576944    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576948    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576951    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576954    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576957    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576960    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576963    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576967    2562 flags.go:64] FLAG: --cgroup-root=""
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576970    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576973    2562 flags.go:64] FLAG: --client-ca-file=""
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576976    2562 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576979    2562 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576983    2562 flags.go:64] FLAG: --cluster-domain=""
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576987    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 18:00:39.582508 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576990    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576993    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.576997    2562 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577000    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577003    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577006    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577009    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577012    2562 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577017    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577020    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577023    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577026    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577030    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577033    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577036    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577039    2562 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577041    2562 flags.go:64] FLAG: --enable-server="true"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577045    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577049    2562 flags.go:64] FLAG: --event-burst="100"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577052    2562 flags.go:64] FLAG: --event-qps="50"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577055    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577058    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577060    2562 flags.go:64] FLAG: --eviction-hard=""
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577064    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 18:00:39.583096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577066    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577069    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577072    2562 flags.go:64] FLAG: --eviction-soft=""
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577075    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577078    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577081    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577084    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577087    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577089    2562 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577092    2562 flags.go:64] FLAG: --feature-gates=""
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577096    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577111    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577117    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577120    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577123    2562 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577126    2562 flags.go:64] FLAG: --help="false"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577130    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577133    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577136    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577139    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577143    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577146    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577149    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 18:00:39.583709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577152    2562 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577155    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577157    2562 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577160    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577164    2562 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577166    2562 flags.go:64] FLAG: --kube-reserved=""
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577169    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577172    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577175    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577177    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577180    2562 flags.go:64] FLAG: --lock-file=""
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577183    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577186    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577188    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577194    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577197    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577199    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577202    2562 flags.go:64] FLAG: --logging-format="text"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577205    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577208    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577211    2562 flags.go:64] FLAG: --manifest-url=""
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577215    2562 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577219    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577221    2562 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577226    2562 flags.go:64] FLAG: --max-pods="110"
Aug 04 18:00:39.584333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577230    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577233    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577235    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577238    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577241    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577244    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577247    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577253    2562 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577256    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577259    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577262    2562 flags.go:64] FLAG: --pod-cidr=""
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577265    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577270    2562 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577273    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577276    2562 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577279    2562 flags.go:64] FLAG: --port="10250"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577281    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577284    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c68aebae0d6bf70e"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577288    2562 flags.go:64] FLAG: --qos-reserved=""
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577291    2562 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577294    2562 flags.go:64] FLAG: --register-node="true"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577296    2562 flags.go:64] FLAG: --register-with-taints=""
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577300    2562 flags.go:64] FLAG: --registry-burst="10"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577303    2562 flags.go:64] FLAG: --registry-qps="5"
Aug 04 18:00:39.584969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577305    2562 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577308    2562 flags.go:64] FLAG: --reserved-memory=""
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577312    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577314    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577319    2562 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577322    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577325    2562 flags.go:64] FLAG: --runonce="false"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577327    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577330    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577335    2562 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577337    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577340    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577344    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577347    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577350    2562 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577352    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577355    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577358    2562 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577361    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577364    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577367    2562 flags.go:64] FLAG: --system-cgroups=""
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577370    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577375    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577378    2562 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577381    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 18:00:39.585570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577385    2562 flags.go:64] FLAG: --tls-min-version=""
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577388    2562 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577391    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577393    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577396    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577399    2562 flags.go:64] FLAG: --v="2"
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577402    2562 flags.go:64] FLAG: --version="false"
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577406    2562 flags.go:64] FLAG: --vmodule=""
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577409    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.577413    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577518    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577523    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577526    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577529    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577531    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577534    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577538    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577540    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577543    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577546    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577548    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577551    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:00:39.586197 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577554    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577556    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577559    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577561    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577564    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577566    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577570    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577572    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577574    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577577    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577579    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577582    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577584    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577587    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577589    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577593    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577596    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577599    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577602    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:00:39.586790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577604    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577607    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577611    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577614    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577617    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577619    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577622    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577624    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577627    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577629    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577632    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577634    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577636    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577639    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577642    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577644    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577646    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577649    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577652    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577654    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:00:39.587278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577657    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577660    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577662    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577665    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577668    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577670    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577673    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577675    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577678    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577680    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577683    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577685    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577688    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577690    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577694    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577697    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577699    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577702    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577704    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577707    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:00:39.587768 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577709    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577712    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577715    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577717    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577720    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577722    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577726    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577730    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577732    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577735    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577738    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577740    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577743    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577745    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577748    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577751    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577755    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577758    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577761    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:00:39.588339 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577763    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577766    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577769    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577772    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577775    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577777    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577780    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577784    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.577787    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.578437    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.584873    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.584888    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.584906    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.584914    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584968    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:00:39.588828 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584972    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584976    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584980    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584983    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584986    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584990    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584992    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584995    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.584998    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585001    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585004    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585007    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585010    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585013    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585015    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585018    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585021    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585023    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585026    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585029    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:00:39.589351 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585031    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585034    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585036    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585039    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585042    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585044    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585047    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585049    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585053    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585056    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585059    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585062    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585064    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585067    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585071    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585075    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585078    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585081    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585084    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:00:39.589985 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585087    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585090    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585092    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585095    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585098    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585118    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585122    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585126    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585128    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585131    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585134    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585137    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585139    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585142    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585145    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585147    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585150    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585152    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585156    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:00:39.590511 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585160    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585163    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585167    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585170    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585172    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585175    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585178    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585180    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585183    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585185    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585188    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585190    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585193    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585195    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585198    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585200    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585204    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585206    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585209    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585212    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:00:39.591143 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585214    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585217    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585220    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585222    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585225    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585228    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585230    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585233    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585235    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585238    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585240    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585243    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585246    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585248    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585251    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585254    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585257    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585259    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585262    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:00:39.591933 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585265    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.585270    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585375    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585379    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585382    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585385    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585388    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585391    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585394    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585396    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585399    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585402    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585404    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585407    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585410    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:00:39.592433 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585413    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585415    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585418    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585420    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585423    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585426    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585428    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585431    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585434    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585437    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585439    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585442    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585444    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585447    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585449    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585452    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585454    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585457    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585459    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585462    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:00:39.593022 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585464    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585467    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585469    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585472    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585474    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585477    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585480    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585482    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585484    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585487    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585489    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585494    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585497    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585500    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585503    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585506    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585509    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585511    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585514    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585516    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:00:39.593809 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585519    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585522    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585525    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585527    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585530    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585532    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585535    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585538    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585540    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585543    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585545    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585548    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585550    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585553    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585555    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585558    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585560    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585563    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585567    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:00:39.594628 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585570    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585574    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585576    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585579    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585582    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585585    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585588    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585591    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585594    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585596    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585599    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585601    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585604    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585607    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585609    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585612    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585616    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585619    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585622    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:00:39.595331 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585625    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585628    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585630    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585633    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585635    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585638    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585640    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:39.585643    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.585647    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.585734    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.587882    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.589234    2562 server.go:1019] "Starting client certificate rotation"
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.589351    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 18:00:39.595810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.589401    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 18:00:39.596222 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.591661    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 18:00:39.596964 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.596946    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 18:00:39.607715 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.607694    2562 log.go:25] "Validated CRI v1 runtime API"
Aug 04 18:00:39.609716 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.609700    2562 log.go:25] "Validated CRI v1 image API"
Aug 04 18:00:39.611195 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.611181    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 18:00:39.613365 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.613349    2562 fs.go:135] Filesystem UUIDs: map[6dc3e9e2-540c-4dff-b6ff-2fff44311a67:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d7bde81f-bd3d-454e-a6f0-366e39e85bc9:/dev/nvme0n1p4]
Aug 04 18:00:39.613415 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.613365    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 18:00:39.618184 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.618071    2562 manager.go:216] Machine: {Timestamp:2026-08-04 18:00:39.616904697 +0000 UTC m=+0.307752430 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100017 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25b6774264667bcaa5ce5177c065ff SystemUUID:ec25b677-4264-667b-caa5-ce5177c065ff BootID:b5c23a9b-1d1b-457e-a527-a1d94e869bf6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2f:e1:ea:3e:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2f:e1:ea:3e:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:a3:d7:8a:a1:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 18:00:39.618184 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.618181    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 18:00:39.618292 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.618253    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 18:00:39.619135 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.619111    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 18:00:39.619268 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.619137    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-239.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 18:00:39.620340 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.620330    2562 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 18:00:39.620378 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.620343    2562 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 18:00:39.620378 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.620350    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 18:00:39.620744 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.620730    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 18:00:39.620941 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.620931    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 18:00:39.621661 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.621652    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 18:00:39.624635 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.624624    2562 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 18:00:39.624735 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.624727    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 18:00:39.624775 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.624754    2562 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 18:00:39.624775 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.624764    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 18:00:39.624834 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.624777    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 18:00:39.624834 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.624787    2562 kubelet.go:398] "Adding apiserver pod source"
Aug 04 18:00:39.624834 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.624795    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 18:00:39.625130 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.625094    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 18:00:39.625170 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.625131    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 18:00:39.626920 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.626898    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 18:00:39.628313 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.628292    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 18:00:39.628362 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.628330    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 18:00:39.629369 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629354    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 18:00:39.629369 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629372    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629378    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629384    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629393    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629399    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629406    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629411    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629418    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629423    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629430    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629435    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629456    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 18:00:39.629478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.629461    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 18:00:39.632657 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.632642    2562 server.go:1268] "Started kubelet"
Aug 04 18:00:39.632755 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.632706    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 18:00:39.632826 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.632716    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 18:00:39.632826 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.632799    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 18:00:39.633384 ip-10-0-136-239 systemd[1]: Started Kubernetes Kubelet.
Aug 04 18:00:39.637051 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.637026    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 18:00:39.638644 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.638601    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 18:00:39.639070 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.639052    2562 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 18:00:39.639857 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.639841    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-239.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 18:00:39.639954 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.639928    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-239.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 18:00:39.639954 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.639943    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 18:00:39.643139 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643118    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 18:00:39.643250 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643135    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 18:00:39.643341 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.643273    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:39.643605 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.642724    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-239.ec2.internal.18c8ac109e97f3ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-239.ec2.internal,UID:ip-10-0-136-239.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-239.ec2.internal,},FirstTimestamp:2026-08-04 18:00:39.632614399 +0000 UTC m=+0.323462131,LastTimestamp:2026-08-04 18:00:39.632614399 +0000 UTC m=+0.323462131,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-239.ec2.internal,}"
Aug 04 18:00:39.643690 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643666    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 18:00:39.643690 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643677    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 18:00:39.643783 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643688    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 18:00:39.643783 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643781    2562 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 18:00:39.643874 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643787    2562 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 18:00:39.643923 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643881    2562 factory.go:55] Registering systemd factory
Aug 04 18:00:39.643923 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.643896    2562 factory.go:223] Registration of the systemd container factory successfully
Aug 04 18:00:39.644186 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.644162    2562 factory.go:153] Registering CRI-O factory
Aug 04 18:00:39.644186 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.644180    2562 factory.go:223] Registration of the crio container factory successfully
Aug 04 18:00:39.644294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.644229    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 18:00:39.644294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.644247    2562 factory.go:103] Registering Raw factory
Aug 04 18:00:39.644294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.644259    2562 manager.go:1196] Started watching for new ooms in manager
Aug 04 18:00:39.644654 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.644642    2562 manager.go:349] Starting recovery of all containers
Aug 04 18:00:39.649559 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.649531    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lsxl5"
Aug 04 18:00:39.651914 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.651889    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 04 18:00:39.652242 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.652057    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-239.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 04 18:00:39.657056 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.657037    2562 manager.go:354] Recovery completed
Aug 04 18:00:39.662564 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.662553    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:00:39.664615 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.664595    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:00:39.664692 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.664624    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:00:39.664692 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.664634    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:00:39.665128 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.665114    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 18:00:39.665186 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.665144    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 18:00:39.665186 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.665159    2562 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 18:00:39.666182 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.666168    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:00:39.666286 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.666274    2562 policy_none.go:49] "None policy: Start"
Aug 04 18:00:39.666812 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.666793    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 18:00:39.666897 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.666826    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 18:00:39.666946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.666898    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:00:39.666946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.666937    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:00:39.667020 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.666950    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:00:39.667743 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.667732    2562 policy_none.go:47] "Start"
Aug 04 18:00:39.671757 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.671738    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lsxl5"
Aug 04 18:00:39.672426 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.672344    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-239.ec2.internal.18c8ac10a08036ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-239.ec2.internal,UID:ip-10-0-136-239.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-239.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-239.ec2.internal,},FirstTimestamp:2026-08-04 18:00:39.66461307 +0000 UTC m=+0.355460805,LastTimestamp:2026-08-04 18:00:39.66461307 +0000 UTC m=+0.355460805,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-239.ec2.internal,}"
Aug 04 18:00:39.701739 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.701724    2562 manager.go:337] "Starting Device Plugin manager"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.701757    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.701769    2562 server.go:87] "Starting device plugin registration server"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.702003    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.702013    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.702122    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.702196    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.702204    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.703380    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 18:00:39.709166 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.703423    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:39.792235 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.792211    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 18:00:39.793499 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.793479    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 18:00:39.793577 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.793517    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 18:00:39.793577 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.793536    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 18:00:39.793680 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.793574    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 18:00:39.795779 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.795762    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:00:39.802395 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.802382    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:00:39.803174 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.803159    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:00:39.803243 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.803188    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:00:39.803243 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.803198    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:00:39.803243 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.803219    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.814834 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.814818    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.814922 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.814840    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-239.ec2.internal\": node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:39.830367 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.830350    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:39.893653 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.893622    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal"]
Aug 04 18:00:39.893761 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.893723    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:00:39.894559 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.894537    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:00:39.894650 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.894575    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:00:39.894650 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.894589    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:00:39.895837 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.895823    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:00:39.895948 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.895934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.895993 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.895969    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:00:39.896823 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.896798    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:00:39.896922 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.896826    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:00:39.896922 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.896840    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:00:39.896922 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.896862    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:00:39.896922 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.896881    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:00:39.896922 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.896889    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:00:39.897945 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.897933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.897987 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.897954    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:00:39.898576 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.898563    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:00:39.898633 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.898587    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:00:39.898633 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.898597    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:00:39.923595 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.923577    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-239.ec2.internal\" not found" node="ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.927731 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.927717    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-239.ec2.internal\" not found" node="ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.930769 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:39.930756    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:39.945582 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.945564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05ea59b9dc8e5b6da11a08076dd1a6df-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal\" (UID: \"05ea59b9dc8e5b6da11a08076dd1a6df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.945637 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.945588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05ea59b9dc8e5b6da11a08076dd1a6df-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal\" (UID: \"05ea59b9dc8e5b6da11a08076dd1a6df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:39.945637 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:39.945605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b6a804075f7c4a3a5c87d537eea8844-config\") pod \"kube-apiserver-proxy-ip-10-0-136-239.ec2.internal\" (UID: \"3b6a804075f7c4a3a5c87d537eea8844\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.030804 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:40.030784    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:40.045801 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.045775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b6a804075f7c4a3a5c87d537eea8844-config\") pod \"kube-apiserver-proxy-ip-10-0-136-239.ec2.internal\" (UID: \"3b6a804075f7c4a3a5c87d537eea8844\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.045867 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.045806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05ea59b9dc8e5b6da11a08076dd1a6df-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal\" (UID: \"05ea59b9dc8e5b6da11a08076dd1a6df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.045867 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.045833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05ea59b9dc8e5b6da11a08076dd1a6df-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal\" (UID: \"05ea59b9dc8e5b6da11a08076dd1a6df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.045946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.045871    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05ea59b9dc8e5b6da11a08076dd1a6df-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal\" (UID: \"05ea59b9dc8e5b6da11a08076dd1a6df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.045946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.045878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b6a804075f7c4a3a5c87d537eea8844-config\") pod \"kube-apiserver-proxy-ip-10-0-136-239.ec2.internal\" (UID: \"3b6a804075f7c4a3a5c87d537eea8844\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.045946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.045921    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05ea59b9dc8e5b6da11a08076dd1a6df-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal\" (UID: \"05ea59b9dc8e5b6da11a08076dd1a6df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.131227 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:40.131167    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:40.225600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.225580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.230810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.230793    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:40.231876 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:40.231859    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:40.332823 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:40.332790    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:40.433298 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:40.433240    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:40.487813 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.487794    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:00:40.533472 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:40.533453    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:40.589008 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.588986    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 18:00:41.597823 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.589154    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 18:00:41.597823 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:40.589155    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 18:00:41.641919 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:41.641896    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:41.658337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.658318    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:00:41.683982 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.683946    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 17:55:40 +0000 UTC" deadline="2028-02-10 04:42:27.550019679 +0000 UTC"
Aug 04 18:00:41.683982 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.683979    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13306h41m45.86604488s"
Aug 04 18:00:41.742380 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:41.742316    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-239.ec2.internal\" not found"
Aug 04 18:00:41.745023 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.745006    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:00:41.752624 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.752609    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:41.764864 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.764841    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 18:00:41.765665 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.765650    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal"
Aug 04 18:00:41.777015 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.776994    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 18:00:41.887922 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:41.887888    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6a804075f7c4a3a5c87d537eea8844.slice/crio-b187c39b0cdac70030ab0dea6087eab2a559a4b05f899f5644e7222710a45e1d WatchSource:0}: Error finding container b187c39b0cdac70030ab0dea6087eab2a559a4b05f899f5644e7222710a45e1d: Status 404 returned error can't find the container with id b187c39b0cdac70030ab0dea6087eab2a559a4b05f899f5644e7222710a45e1d
Aug 04 18:00:41.888395 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:41.888370    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ea59b9dc8e5b6da11a08076dd1a6df.slice/crio-06d47911f2ee0f8b2a404ee7ebf6736a8e607bb82d8b82e3155a9e3ff17be97b WatchSource:0}: Error finding container 06d47911f2ee0f8b2a404ee7ebf6736a8e607bb82d8b82e3155a9e3ff17be97b: Status 404 returned error can't find the container with id 06d47911f2ee0f8b2a404ee7ebf6736a8e607bb82d8b82e3155a9e3ff17be97b
Aug 04 18:00:41.891846 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:41.891832    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:00:42.634098 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.634058    2562 apiserver.go:52] "Watching apiserver"
Aug 04 18:00:42.643480 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.643452    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 18:00:42.645594 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.645568    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-qw958","openshift-multus/multus-x9p9n","openshift-multus/network-metrics-daemon-gw46q","openshift-network-diagnostics/network-check-target-rvmpt","openshift-network-operator/iptables-alerter-mczn8","kube-system/konnectivity-agent-jcl8j","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj","openshift-cluster-node-tuning-operator/tuned-vx46c","openshift-dns/node-resolver-vt5k6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-fnkpr","kube-system/global-pull-secret-syncer-2qbzg","kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal","openshift-image-registry/node-ca-kkvwm"]
Aug 04 18:00:42.647000 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.646979    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.648734 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.648680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.649862 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.649841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:42.649968 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.649911    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:42.651081 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.650974    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:42.651081 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.651034    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:42.652205 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.652185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.652489 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.652473    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 18:00:42.652730 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.652713    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 18:00:42.652806 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.652721    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 18:00:42.652806 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.652795    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 18:00:42.653711 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.653460    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.654552 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.654210    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7v2s4\""
Aug 04 18:00:42.654843 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.654826    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mnkb8\""
Aug 04 18:00:42.655656 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.655449    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 18:00:42.655656 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.655566    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 18:00:42.655801 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.655698    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 18:00:42.655801 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.655741    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.656806 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.656785    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 18:00:42.657413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657025    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 18:00:42.657413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657107    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j5szz\""
Aug 04 18:00:42.657413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657256    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 18:00:42.657413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657278    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 18:00:42.657675 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657415    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 18:00:42.657675 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657469    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:00:42.657675 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.657675 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657637    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 18:00:42.657859 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657688    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 18:00:42.657859 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.657757    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7mgmx\""
Aug 04 18:00:42.658473 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.658349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ttb85\""
Aug 04 18:00:42.658593 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.658581    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.659334 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.659131    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 18:00:42.659334 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.659306    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 18:00:42.659990 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.659976    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.660871 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.660852    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 18:00:42.661353 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.661334    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.661439 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.661335    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rnb46\""
Aug 04 18:00:42.662609 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662590    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.662702 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.662702 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.662657    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:42.662702 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m766\" (UniqueName: \"kubernetes.io/projected/4ce8f519-594e-43eb-9e6e-45143be8e3c8-kube-api-access-5m766\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:42.662870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-socket-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.662870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-etc-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.662870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662764    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.662870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwc7x\" (UniqueName: \"kubernetes.io/projected/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-kube-api-access-pwc7x\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.662870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662819    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-kubelet\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.662870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662841    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-ovn\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.662870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662885    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-env-overrides\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e6705ffb-493b-401b-92f7-2ead628737a3-iptables-alerter-script\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6705ffb-493b-401b-92f7-2ead628737a3-host-slash\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662966    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdn22\" (UniqueName: \"kubernetes.io/projected/e6705ffb-493b-401b-92f7-2ead628737a3-kube-api-access-gdn22\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.662987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-run-netns\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-var-lib-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663017    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663037    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43e87821-b4a3-4117-84cd-3f5410422a03-konnectivity-ca\") pod \"konnectivity-agent-jcl8j\" (UID: \"43e87821-b4a3-4117-84cd-3f5410422a03\") " pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-sys-fs\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663132    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-systemd\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-cni-bin\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-cni-netd\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovnkube-config\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77wvw\" (UniqueName: \"kubernetes.io/projected/4f1edb22-437e-44e2-96b1-f7de7c538fe6-kube-api-access-77wvw\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-registration-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-device-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-systemd-units\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-node-log\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-slash\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-log-socket\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovnkube-script-lib\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663467    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:42.663895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.663490    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43e87821-b4a3-4117-84cd-3f5410422a03-agent-certs\") pod \"konnectivity-agent-jcl8j\" (UID: \"43e87821-b4a3-4117-84cd-3f5410422a03\") " pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.665179 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.664998    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 18:00:42.665179 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.665008    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 18:00:42.665320 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.665223    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.665707    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.665924    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.666253    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.666428    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.666586    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nd8mb\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.666622    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.666637    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 18:00:42.666838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.666754    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pwk2m\""
Aug 04 18:00:42.667260 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.666878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jxmhw\""
Aug 04 18:00:42.667689 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.667669    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 18:00:42.678851 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.678825    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 18:00:42.708330 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.708254    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xgw6q"
Aug 04 18:00:42.717919 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.717805    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xgw6q"
Aug 04 18:00:42.753116 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.753094    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 18:00:42.764628 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-socket-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.764736 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764632    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-etc-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.764736 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysconfig\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.764736 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.764736 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-conf-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drl6t\" (UniqueName: \"kubernetes.io/projected/d854e521-0559-4697-9e68-a670969d3df1-kube-api-access-drl6t\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-host\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-ovn\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-ovn\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-etc-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764836    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e6705ffb-493b-401b-92f7-2ead628737a3-iptables-alerter-script\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764784    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-socket-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdn22\" (UniqueName: \"kubernetes.io/projected/e6705ffb-493b-401b-92f7-2ead628737a3-kube-api-access-gdn22\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.764907 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764886    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764915    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-system-cni-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764941    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-tuned\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.764985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfqfm\" (UniqueName: \"kubernetes.io/projected/5204927c-d0ff-4e9b-830e-f06d22586a17-kube-api-access-cfqfm\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765008    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-tmp\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765041    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-kubelet\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-sys-fs\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-systemd\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-cni-netd\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovnkube-config\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765209    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77wvw\" (UniqueName: \"kubernetes.io/projected/4f1edb22-437e-44e2-96b1-f7de7c538fe6-kube-api-access-77wvw\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-sys-fs\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765285    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-run-systemd\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-cni-netd\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765330    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e6705ffb-493b-401b-92f7-2ead628737a3-iptables-alerter-script\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.765414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4681ca7-7610-4a70-951f-850e67bb9c1d-hosts-file\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-hostroot\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-registration-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-device-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-systemd-units\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-node-log\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765521    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-registration-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-cni-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-systemd-units\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765595    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-node-log\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-etc-kubernetes\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765623    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5204927c-d0ff-4e9b-830e-f06d22586a17-host\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-device-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-os-release\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-slash\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-slash\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-log-socket\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-log-socket\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovnkube-script-lib\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovnkube-config\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8hng\" (UniqueName: \"kubernetes.io/projected/b4681ca7-7610-4a70-951f-850e67bb9c1d-kube-api-access-w8hng\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765891    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-cnibin\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-modprobe-d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.765924    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.765996    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs podName:4ce8f519-594e-43eb-9e6e-45143be8e3c8 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:43.265960262 +0000 UTC m=+2.948408581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs") pod "network-metrics-daemon-gw46q" (UID: "4ce8f519-594e-43eb-9e6e-45143be8e3c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.765999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766015    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysctl-d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766042    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-run\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-systemd\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-lib-modules\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.766995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2g4d\" (UniqueName: \"kubernetes.io/projected/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-kube-api-access-b2g4d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766149    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766165    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-system-cni-dir\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5nqv\" (UniqueName: \"kubernetes.io/projected/841bbbda-ca31-4cdf-8374-929647c20747-kube-api-access-n5nqv\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e5b59001-bd6e-4030-83f7-0d46dcc42501-kubelet-config\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-k8s-cni-cncf-io\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766290    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovnkube-script-lib\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-multus-certs\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-kubernetes\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-sys\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766397    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-var-lib-kubelet\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766419    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e5b59001-bd6e-4030-83f7-0d46dcc42501-dbus\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-cni-bin\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-cni-multus\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwc7x\" (UniqueName: \"kubernetes.io/projected/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-kube-api-access-pwc7x\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.767749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-kubelet\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766579    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-env-overrides\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6705ffb-493b-401b-92f7-2ead628737a3-host-slash\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43e87821-b4a3-4117-84cd-3f5410422a03-konnectivity-ca\") pod \"konnectivity-agent-jcl8j\" (UID: \"43e87821-b4a3-4117-84cd-3f5410422a03\") " pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5204927c-d0ff-4e9b-830e-f06d22586a17-serviceca\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766672    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-netns\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-run-netns\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766739    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-var-lib-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-kubelet\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-socket-dir-parent\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-run-netns\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.766928    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-var-lib-openvswitch\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f1edb22-437e-44e2-96b1-f7de7c538fe6-env-overrides\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767101    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6705ffb-493b-401b-92f7-2ead628737a3-host-slash\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysctl-conf\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-cnibin\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.768438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-cni-bin\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767223    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d854e521-0559-4697-9e68-a670969d3df1-multus-daemon-config\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4681ca7-7610-4a70-951f-850e67bb9c1d-tmp-dir\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f1edb22-437e-44e2-96b1-f7de7c538fe6-host-cni-bin\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d854e521-0559-4697-9e68-a670969d3df1-cni-binary-copy\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-os-release\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-cni-binary-copy\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43e87821-b4a3-4117-84cd-3f5410422a03-konnectivity-ca\") pod \"konnectivity-agent-jcl8j\" (UID: \"43e87821-b4a3-4117-84cd-3f5410422a03\") " pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767447    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43e87821-b4a3-4117-84cd-3f5410422a03-agent-certs\") pod \"konnectivity-agent-jcl8j\" (UID: \"43e87821-b4a3-4117-84cd-3f5410422a03\") " pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767575    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.769169 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.767575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m766\" (UniqueName: \"kubernetes.io/projected/4ce8f519-594e-43eb-9e6e-45143be8e3c8-kube-api-access-5m766\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:42.771716 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.771695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43e87821-b4a3-4117-84cd-3f5410422a03-agent-certs\") pod \"konnectivity-agent-jcl8j\" (UID: \"43e87821-b4a3-4117-84cd-3f5410422a03\") " pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.771852 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.771836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f1edb22-437e-44e2-96b1-f7de7c538fe6-ovn-node-metrics-cert\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.779754 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.779734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdn22\" (UniqueName: \"kubernetes.io/projected/e6705ffb-493b-401b-92f7-2ead628737a3-kube-api-access-gdn22\") pod \"iptables-alerter-mczn8\" (UID: \"e6705ffb-493b-401b-92f7-2ead628737a3\") " pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.783314 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.783274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwc7x\" (UniqueName: \"kubernetes.io/projected/c4ea25f0-614d-4356-ab3f-e79c4cf3b15d-kube-api-access-pwc7x\") pod \"aws-ebs-csi-driver-node-vvrwj\" (UID: \"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.783810 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.783614    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:00:42.783810 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.783635    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:00:42.783810 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.783647    2562 projected.go:196] Error preparing data for projected volume kube-api-access-5gttp for pod openshift-network-diagnostics/network-check-target-rvmpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:42.783810 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.783716    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp podName:d5691a2d-bea0-40f6-a2d2-6415a59a4c93 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:43.28370065 +0000 UTC m=+2.966148971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5gttp" (UniqueName: "kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp") pod "network-check-target-rvmpt" (UID: "d5691a2d-bea0-40f6-a2d2-6415a59a4c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:42.785319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.785298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77wvw\" (UniqueName: \"kubernetes.io/projected/4f1edb22-437e-44e2-96b1-f7de7c538fe6-kube-api-access-77wvw\") pod \"ovnkube-node-fnkpr\" (UID: \"4f1edb22-437e-44e2-96b1-f7de7c538fe6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.785636 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.785620    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m766\" (UniqueName: \"kubernetes.io/projected/4ce8f519-594e-43eb-9e6e-45143be8e3c8-kube-api-access-5m766\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:42.808856 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.808808    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal" event={"ID":"05ea59b9dc8e5b6da11a08076dd1a6df","Type":"ContainerStarted","Data":"06d47911f2ee0f8b2a404ee7ebf6736a8e607bb82d8b82e3155a9e3ff17be97b"}
Aug 04 18:00:42.810023 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.810000    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal" event={"ID":"3b6a804075f7c4a3a5c87d537eea8844","Type":"ContainerStarted","Data":"b187c39b0cdac70030ab0dea6087eab2a559a4b05f899f5644e7222710a45e1d"}
Aug 04 18:00:42.867926 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.867902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysctl-d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.867937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-run\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.867956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-systemd\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.867969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-lib-modules\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.867984    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2g4d\" (UniqueName: \"kubernetes.io/projected/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-kube-api-access-b2g4d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-system-cni-dir\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.868047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5nqv\" (UniqueName: \"kubernetes.io/projected/841bbbda-ca31-4cdf-8374-929647c20747-kube-api-access-n5nqv\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.868047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e5b59001-bd6e-4030-83f7-0d46dcc42501-kubelet-config\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868063    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-k8s-cni-cncf-io\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-multus-certs\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysctl-d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-kubernetes\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868147    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-sys\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868161    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-systemd\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868170    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-var-lib-kubelet\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e5b59001-bd6e-4030-83f7-0d46dcc42501-kubelet-config\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e5b59001-bd6e-4030-83f7-0d46dcc42501-dbus\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-system-cni-dir\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.868208    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-cni-bin\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-multus-certs\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-cni-multus\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-lib-modules\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.868420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-kubernetes\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:42.868277    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret podName:e5b59001-bd6e-4030-83f7-0d46dcc42501 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:43.368258239 +0000 UTC m=+3.050706539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret") pod "global-pull-secret-syncer-2qbzg" (UID: "e5b59001-bd6e-4030-83f7-0d46dcc42501") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-run\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-cni-multus\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868301    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-k8s-cni-cncf-io\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868281    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-cni-bin\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868356    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-sys\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5204927c-d0ff-4e9b-830e-f06d22586a17-serviceca\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-var-lib-kubelet\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-netns\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-socket-dir-parent\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e5b59001-bd6e-4030-83f7-0d46dcc42501-dbus\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-run-netns\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868458    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysctl-conf\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-socket-dir-parent\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-cnibin\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d854e521-0559-4697-9e68-a670969d3df1-multus-daemon-config\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.869287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4681ca7-7610-4a70-951f-850e67bb9c1d-tmp-dir\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d854e521-0559-4697-9e68-a670969d3df1-cni-binary-copy\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868603    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-cnibin\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868620    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysctl-conf\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-os-release\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-cni-binary-copy\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868816    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysconfig\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5204927c-d0ff-4e9b-830e-f06d22586a17-serviceca\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-conf-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-os-release\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.868908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4681ca7-7610-4a70-951f-850e67bb9c1d-tmp-dir\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869064    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-sysconfig\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d854e521-0559-4697-9e68-a670969d3df1-multus-daemon-config\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869279    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-conf-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/841bbbda-ca31-4cdf-8374-929647c20747-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drl6t\" (UniqueName: \"kubernetes.io/projected/d854e521-0559-4697-9e68-a670969d3df1-kube-api-access-drl6t\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-host\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-system-cni-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869385    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-tuned\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfqfm\" (UniqueName: \"kubernetes.io/projected/5204927c-d0ff-4e9b-830e-f06d22586a17-kube-api-access-cfqfm\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-tmp\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-kubelet\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869453    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-system-cni-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-cni-binary-copy\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4681ca7-7610-4a70-951f-850e67bb9c1d-hosts-file\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/841bbbda-ca31-4cdf-8374-929647c20747-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-host-var-lib-kubelet\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-host\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-hostroot\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869584    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4681ca7-7610-4a70-951f-850e67bb9c1d-hosts-file\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869596    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-cni-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.870688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-etc-kubernetes\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-hostroot\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869650    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5204927c-d0ff-4e9b-830e-f06d22586a17-host\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-os-release\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-multus-cni-dir\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-etc-kubernetes\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5204927c-d0ff-4e9b-830e-f06d22586a17-host\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8hng\" (UniqueName: \"kubernetes.io/projected/b4681ca7-7610-4a70-951f-850e67bb9c1d-kube-api-access-w8hng\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869748    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-os-release\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-cnibin\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-modprobe-d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869860    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d854e521-0559-4697-9e68-a670969d3df1-cnibin\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d854e521-0559-4697-9e68-a670969d3df1-cni-binary-copy\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.871280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.869980    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-modprobe-d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.871707 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.871570    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-etc-tuned\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.871707 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.871602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-tmp\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.883035 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.883016    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5nqv\" (UniqueName: \"kubernetes.io/projected/841bbbda-ca31-4cdf-8374-929647c20747-kube-api-access-n5nqv\") pod \"multus-additional-cni-plugins-qw958\" (UID: \"841bbbda-ca31-4cdf-8374-929647c20747\") " pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:42.888539 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.888474    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfqfm\" (UniqueName: \"kubernetes.io/projected/5204927c-d0ff-4e9b-830e-f06d22586a17-kube-api-access-cfqfm\") pod \"node-ca-kkvwm\" (UID: \"5204927c-d0ff-4e9b-830e-f06d22586a17\") " pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:42.891678 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.891655    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2g4d\" (UniqueName: \"kubernetes.io/projected/6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9-kube-api-access-b2g4d\") pod \"tuned-vx46c\" (UID: \"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9\") " pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:42.892211 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.892188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8hng\" (UniqueName: \"kubernetes.io/projected/b4681ca7-7610-4a70-951f-850e67bb9c1d-kube-api-access-w8hng\") pod \"node-resolver-vt5k6\" (UID: \"b4681ca7-7610-4a70-951f-850e67bb9c1d\") " pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.893115 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.893097    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drl6t\" (UniqueName: \"kubernetes.io/projected/d854e521-0559-4697-9e68-a670969d3df1-kube-api-access-drl6t\") pod \"multus-x9p9n\" (UID: \"d854e521-0559-4697-9e68-a670969d3df1\") " pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:42.958627 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.958585    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj"
Aug 04 18:00:42.966958 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.966937    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:00:42.977614 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.977594    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mczn8"
Aug 04 18:00:42.984118 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.984100    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:00:42.991645 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.991618    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vt5k6"
Aug 04 18:00:42.999181 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:42.999155    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kkvwm"
Aug 04 18:00:43.006671 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.006651    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x9p9n"
Aug 04 18:00:43.015120 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.015098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vx46c"
Aug 04 18:00:43.020645 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.020623    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qw958"
Aug 04 18:00:43.042869 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.042850    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:00:43.272191 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.272161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:43.272347 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.272296    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:43.272395 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.272359    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs podName:4ce8f519-594e-43eb-9e6e-45143be8e3c8 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:44.27234012 +0000 UTC m=+3.954788433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs") pod "network-metrics-daemon-gw46q" (UID: "4ce8f519-594e-43eb-9e6e-45143be8e3c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:43.372797 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.372760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:43.372959 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.372808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:43.372959 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.372932    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:43.372959 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.372942    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:00:43.372959 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.372956    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:00:43.373160 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.372968    2562 projected.go:196] Error preparing data for projected volume kube-api-access-5gttp for pod openshift-network-diagnostics/network-check-target-rvmpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:43.373160 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.373004    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret podName:e5b59001-bd6e-4030-83f7-0d46dcc42501 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:44.372983343 +0000 UTC m=+4.055431639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret") pod "global-pull-secret-syncer-2qbzg" (UID: "e5b59001-bd6e-4030-83f7-0d46dcc42501") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:43.373160 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.373027    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp podName:d5691a2d-bea0-40f6-a2d2-6415a59a4c93 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:44.373014588 +0000 UTC m=+4.055462897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gttp" (UniqueName: "kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp") pod "network-check-target-rvmpt" (UID: "d5691a2d-bea0-40f6-a2d2-6415a59a4c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:43.677745 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.677702    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6885ce7b_c7f5_4abf_8ebf_6a11fa1e55c9.slice/crio-ae89075db5303f88daf01f3fd329f68d93dcc1c2bb366833fbeb71fd1c80e972 WatchSource:0}: Error finding container ae89075db5303f88daf01f3fd329f68d93dcc1c2bb366833fbeb71fd1c80e972: Status 404 returned error can't find the container with id ae89075db5303f88daf01f3fd329f68d93dcc1c2bb366833fbeb71fd1c80e972
Aug 04 18:00:43.679661 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.679632    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841bbbda_ca31_4cdf_8374_929647c20747.slice/crio-59c038e1ffe95f23a6bc541dcce637146773dc46fade14bfa88b6a9fb2705578 WatchSource:0}: Error finding container 59c038e1ffe95f23a6bc541dcce637146773dc46fade14bfa88b6a9fb2705578: Status 404 returned error can't find the container with id 59c038e1ffe95f23a6bc541dcce637146773dc46fade14bfa88b6a9fb2705578
Aug 04 18:00:43.682920 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.682899    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4681ca7_7610_4a70_951f_850e67bb9c1d.slice/crio-8af0a56eb998fe78bb9a889dc151814fa8e4ccb3fb7e62d757c404f8273ec7d8 WatchSource:0}: Error finding container 8af0a56eb998fe78bb9a889dc151814fa8e4ccb3fb7e62d757c404f8273ec7d8: Status 404 returned error can't find the container with id 8af0a56eb998fe78bb9a889dc151814fa8e4ccb3fb7e62d757c404f8273ec7d8
Aug 04 18:00:43.683790 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.683532    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5204927c_d0ff_4e9b_830e_f06d22586a17.slice/crio-e832427918079ab4e28af4e027546c8c61720dd89fa6e5c64e65b9ce1c5b7686 WatchSource:0}: Error finding container e832427918079ab4e28af4e027546c8c61720dd89fa6e5c64e65b9ce1c5b7686: Status 404 returned error can't find the container with id e832427918079ab4e28af4e027546c8c61720dd89fa6e5c64e65b9ce1c5b7686
Aug 04 18:00:43.685009 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.684870    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6705ffb_493b_401b_92f7_2ead628737a3.slice/crio-d2fd32bea3fd5b9b2caf1fa1564be89f6e0910541765bfce4070a93f48552b34 WatchSource:0}: Error finding container d2fd32bea3fd5b9b2caf1fa1564be89f6e0910541765bfce4070a93f48552b34: Status 404 returned error can't find the container with id d2fd32bea3fd5b9b2caf1fa1564be89f6e0910541765bfce4070a93f48552b34
Aug 04 18:00:43.685914 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.685892    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd854e521_0559_4697_9e68_a670969d3df1.slice/crio-5581fffdbaaf0f97eb608d3306760ab9bdfadc2a910c99dc0f6278dbcd85d3ac WatchSource:0}: Error finding container 5581fffdbaaf0f97eb608d3306760ab9bdfadc2a910c99dc0f6278dbcd85d3ac: Status 404 returned error can't find the container with id 5581fffdbaaf0f97eb608d3306760ab9bdfadc2a910c99dc0f6278dbcd85d3ac
Aug 04 18:00:43.686729 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.686623    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e87821_b4a3_4117_84cd_3f5410422a03.slice/crio-dcbb93767a5b5e839c6c89e92fe7513bfc3bc294f1372c58b9ad2c1fc6d4c94d WatchSource:0}: Error finding container dcbb93767a5b5e839c6c89e92fe7513bfc3bc294f1372c58b9ad2c1fc6d4c94d: Status 404 returned error can't find the container with id dcbb93767a5b5e839c6c89e92fe7513bfc3bc294f1372c58b9ad2c1fc6d4c94d
Aug 04 18:00:43.687724 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.687637    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1edb22_437e_44e2_96b1_f7de7c538fe6.slice/crio-c80235f6cb8a6388a823c16d07eb838823eac5d9f08cbfc80354a76c4a624af4 WatchSource:0}: Error finding container c80235f6cb8a6388a823c16d07eb838823eac5d9f08cbfc80354a76c4a624af4: Status 404 returned error can't find the container with id c80235f6cb8a6388a823c16d07eb838823eac5d9f08cbfc80354a76c4a624af4
Aug 04 18:00:43.689571 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:00:43.689484    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ea25f0_614d_4356_ab3f_e79c4cf3b15d.slice/crio-26a6b63f76368ca52e26c7d4bb8558ff0c01322d7ca2ad8388f6da4987785395 WatchSource:0}: Error finding container 26a6b63f76368ca52e26c7d4bb8558ff0c01322d7ca2ad8388f6da4987785395: Status 404 returned error can't find the container with id 26a6b63f76368ca52e26c7d4bb8558ff0c01322d7ca2ad8388f6da4987785395
Aug 04 18:00:43.720330 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.720305    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 17:55:42 +0000 UTC" deadline="2028-03-21 07:50:13.063976244 +0000 UTC"
Aug 04 18:00:43.720330 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.720329    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14269h49m29.34364965s"
Aug 04 18:00:43.777081 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.777063    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:00:43.802405 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.802384    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:43.802495 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.802478    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:43.802554 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.802526    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:43.802632 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:43.802616    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:43.813963 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.813944    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x9p9n" event={"ID":"d854e521-0559-4697-9e68-a670969d3df1","Type":"ContainerStarted","Data":"5581fffdbaaf0f97eb608d3306760ab9bdfadc2a910c99dc0f6278dbcd85d3ac"}
Aug 04 18:00:43.816318 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.816299    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mczn8" event={"ID":"e6705ffb-493b-401b-92f7-2ead628737a3","Type":"ContainerStarted","Data":"d2fd32bea3fd5b9b2caf1fa1564be89f6e0910541765bfce4070a93f48552b34"}
Aug 04 18:00:43.817990 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.817966    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vt5k6" event={"ID":"b4681ca7-7610-4a70-951f-850e67bb9c1d","Type":"ContainerStarted","Data":"8af0a56eb998fe78bb9a889dc151814fa8e4ccb3fb7e62d757c404f8273ec7d8"}
Aug 04 18:00:43.822080 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.822052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal" event={"ID":"3b6a804075f7c4a3a5c87d537eea8844","Type":"ContainerStarted","Data":"8d31cbc25c4238492bbc849c09928a52a9b276067aa80926957f9e06b7871b31"}
Aug 04 18:00:43.823055 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.823038    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj" event={"ID":"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d","Type":"ContainerStarted","Data":"26a6b63f76368ca52e26c7d4bb8558ff0c01322d7ca2ad8388f6da4987785395"}
Aug 04 18:00:43.825129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.825109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jcl8j" event={"ID":"43e87821-b4a3-4117-84cd-3f5410422a03","Type":"ContainerStarted","Data":"dcbb93767a5b5e839c6c89e92fe7513bfc3bc294f1372c58b9ad2c1fc6d4c94d"}
Aug 04 18:00:43.826229 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.826212    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kkvwm" event={"ID":"5204927c-d0ff-4e9b-830e-f06d22586a17","Type":"ContainerStarted","Data":"e832427918079ab4e28af4e027546c8c61720dd89fa6e5c64e65b9ce1c5b7686"}
Aug 04 18:00:43.827288 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.827261    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerStarted","Data":"59c038e1ffe95f23a6bc541dcce637146773dc46fade14bfa88b6a9fb2705578"}
Aug 04 18:00:43.828246 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.828227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vx46c" event={"ID":"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9","Type":"ContainerStarted","Data":"ae89075db5303f88daf01f3fd329f68d93dcc1c2bb366833fbeb71fd1c80e972"}
Aug 04 18:00:43.830361 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.830343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"c80235f6cb8a6388a823c16d07eb838823eac5d9f08cbfc80354a76c4a624af4"}
Aug 04 18:00:43.839977 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:43.839931    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-239.ec2.internal" podStartSLOduration=2.83991981 podStartE2EDuration="2.83991981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:00:43.839221969 +0000 UTC m=+3.521670284" watchObservedRunningTime="2026-08-04 18:00:43.83991981 +0000 UTC m=+3.522368128"
Aug 04 18:00:44.279832 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.279582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:44.280008 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.279866    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:44.280008 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.279940    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs podName:4ce8f519-594e-43eb-9e6e-45143be8e3c8 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:46.279920888 +0000 UTC m=+5.962369203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs") pod "network-metrics-daemon-gw46q" (UID: "4ce8f519-594e-43eb-9e6e-45143be8e3c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.380818    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.380874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.381011    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.381028    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.381036    2562 projected.go:196] Error preparing data for projected volume kube-api-access-5gttp for pod openshift-network-diagnostics/network-check-target-rvmpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.381079    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp podName:d5691a2d-bea0-40f6-a2d2-6415a59a4c93 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:46.381066398 +0000 UTC m=+6.063514694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gttp" (UniqueName: "kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp") pod "network-check-target-rvmpt" (UID: "d5691a2d-bea0-40f6-a2d2-6415a59a4c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.381360    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:44.381419 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.381393    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret podName:e5b59001-bd6e-4030-83f7-0d46dcc42501 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:46.381384113 +0000 UTC m=+6.063832408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret") pod "global-pull-secret-syncer-2qbzg" (UID: "e5b59001-bd6e-4030-83f7-0d46dcc42501") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:44.721055 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.720949    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 17:55:42 +0000 UTC" deadline="2028-03-30 13:49:27.966796412 +0000 UTC"
Aug 04 18:00:44.721055 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.721009    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14491h48m43.245791865s"
Aug 04 18:00:44.805412 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.805382    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:44.805597 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:44.805536    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:44.840894 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.840864    2562 generic.go:356] "Generic (PLEG): container finished" podID="05ea59b9dc8e5b6da11a08076dd1a6df" containerID="e5dbb8b026afb4b97ceb90b1970e96769ae8d0c45c908e11cec92354ad4e71d1" exitCode=0
Aug 04 18:00:44.841715 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:44.841689    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal" event={"ID":"05ea59b9dc8e5b6da11a08076dd1a6df","Type":"ContainerDied","Data":"e5dbb8b026afb4b97ceb90b1970e96769ae8d0c45c908e11cec92354ad4e71d1"}
Aug 04 18:00:45.802879 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:45.802848    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:45.803373 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:45.802982    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:45.803373 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:45.803354    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:45.803487 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:45.803447    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:45.852970 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:45.852939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal" event={"ID":"05ea59b9dc8e5b6da11a08076dd1a6df","Type":"ContainerStarted","Data":"1c00ca67011d60abf2e73975dececfd18a0efb493d845d0e952a96ed87b28fce"}
Aug 04 18:00:46.296635 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:46.296595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:46.296884 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.296811    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:46.296884 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.296875    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs podName:4ce8f519-594e-43eb-9e6e-45143be8e3c8 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:50.296856792 +0000 UTC m=+9.979305105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs") pod "network-metrics-daemon-gw46q" (UID: "4ce8f519-594e-43eb-9e6e-45143be8e3c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:46.397137 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:46.397101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:46.397300 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:46.397166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:46.397373 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.397306    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:00:46.397373 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.397328    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:00:46.397373 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.397342    2562 projected.go:196] Error preparing data for projected volume kube-api-access-5gttp for pod openshift-network-diagnostics/network-check-target-rvmpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:46.397565 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.397397    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp podName:d5691a2d-bea0-40f6-a2d2-6415a59a4c93 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:50.397378256 +0000 UTC m=+10.079826558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gttp" (UniqueName: "kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp") pod "network-check-target-rvmpt" (UID: "d5691a2d-bea0-40f6-a2d2-6415a59a4c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:46.397824 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.397804    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:46.397917 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.397862    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret podName:e5b59001-bd6e-4030-83f7-0d46dcc42501 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:50.397846832 +0000 UTC m=+10.080295132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret") pod "global-pull-secret-syncer-2qbzg" (UID: "e5b59001-bd6e-4030-83f7-0d46dcc42501") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:46.805273 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:46.805231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:46.805753 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:46.805347    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:47.802334 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:47.802303    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:47.802521 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:47.802295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:47.802521 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:47.802430    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:47.802653 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:47.802527    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:48.802346 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:48.802319    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:48.802820 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:48.802445    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:49.802646 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:49.802141    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:49.802646 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:49.802282    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:49.802646 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:49.802450    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:49.802646 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:49.802580    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:50.334467 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:50.334433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:50.334678 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.334622    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:50.334739 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.334683    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs podName:4ce8f519-594e-43eb-9e6e-45143be8e3c8 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:58.334665249 +0000 UTC m=+18.017113546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs") pod "network-metrics-daemon-gw46q" (UID: "4ce8f519-594e-43eb-9e6e-45143be8e3c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:50.434994 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:50.434962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:50.435173 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:50.435021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:50.435236 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.435187    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:00:50.435236 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.435208    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:00:50.435236 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.435220    2562 projected.go:196] Error preparing data for projected volume kube-api-access-5gttp for pod openshift-network-diagnostics/network-check-target-rvmpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:50.435437 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.435272    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp podName:d5691a2d-bea0-40f6-a2d2-6415a59a4c93 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:58.435254284 +0000 UTC m=+18.117702585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gttp" (UniqueName: "kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp") pod "network-check-target-rvmpt" (UID: "d5691a2d-bea0-40f6-a2d2-6415a59a4c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:50.435437 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.435349    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:50.435437 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.435396    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret podName:e5b59001-bd6e-4030-83f7-0d46dcc42501 nodeName:}" failed. No retries permitted until 2026-08-04 18:00:58.435383199 +0000 UTC m=+18.117831498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret") pod "global-pull-secret-syncer-2qbzg" (UID: "e5b59001-bd6e-4030-83f7-0d46dcc42501") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:50.804048 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:50.803705    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:50.804048 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:50.803830    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:51.802561 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:51.802529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:51.802733 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:51.802574    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:51.802733 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:51.802681    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:51.802863 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:51.802836    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:52.802362 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:52.802316    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:52.802823 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:52.802452    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:53.802340 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:53.802311    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:53.802498 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:53.802406    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:53.802498 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:53.802457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:53.802819 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:53.802553    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:54.802723 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:54.802691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:54.803124 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:54.802814    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:55.802560 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:55.802527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:55.802741 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:55.802661    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:55.802741 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:55.802712    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:55.803158 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:55.802812    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:56.802610 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:56.802573    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:56.802801 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:56.802699    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:57.802875 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:57.802842    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:57.803396 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:57.802841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:57.803396 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:57.802974    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:00:57.803396 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:57.803028    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:58.400946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:58.400862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:58.401140 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.401036    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:58.401140 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.401109    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs podName:4ce8f519-594e-43eb-9e6e-45143be8e3c8 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:14.401088872 +0000 UTC m=+34.083537190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs") pod "network-metrics-daemon-gw46q" (UID: "4ce8f519-594e-43eb-9e6e-45143be8e3c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:00:58.502075 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:58.502028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:58.502231 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:58.502103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:58.502231 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.502178    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:58.502231 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.502228    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:00:58.502398 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.502245    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:00:58.502398 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.502250    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret podName:e5b59001-bd6e-4030-83f7-0d46dcc42501 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:14.502229263 +0000 UTC m=+34.184677572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret") pod "global-pull-secret-syncer-2qbzg" (UID: "e5b59001-bd6e-4030-83f7-0d46dcc42501") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:00:58.502398 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.502258    2562 projected.go:196] Error preparing data for projected volume kube-api-access-5gttp for pod openshift-network-diagnostics/network-check-target-rvmpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:58.502398 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.502302    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp podName:d5691a2d-bea0-40f6-a2d2-6415a59a4c93 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:14.502287332 +0000 UTC m=+34.184735649 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gttp" (UniqueName: "kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp") pod "network-check-target-rvmpt" (UID: "d5691a2d-bea0-40f6-a2d2-6415a59a4c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:00:58.802193 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:58.802131    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:00:58.802369 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:58.802279    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:00:59.802270 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:59.802236    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:00:59.802714 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:00:59.802288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:00:59.802714 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:59.802373    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:00:59.802714 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:00:59.802523    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:00.803646 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:00.803617    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:00.804082 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:00.803744    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:01.802438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:01.802410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:01.802604 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:01.802514    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:01.802604 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:01.802558    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:01.802684 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:01.802643    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:01:02.802742 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.802427    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:02.803325 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:02.802832    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:02.886398 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.886330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x9p9n" event={"ID":"d854e521-0559-4697-9e68-a670969d3df1","Type":"ContainerStarted","Data":"6b5d2d0665508dbb34a00eaae1253e2c6d2ebead4efdbdbd848aa46d27d61b27"}
Aug 04 18:01:02.887640 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.887615    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vt5k6" event={"ID":"b4681ca7-7610-4a70-951f-850e67bb9c1d","Type":"ContainerStarted","Data":"34f75b21a8b8e768f67df0e6a867b293e08989b72e53290d61a2023d266a2cf4"}
Aug 04 18:01:02.888985 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.888966    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj" event={"ID":"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d","Type":"ContainerStarted","Data":"041c93577f8d80249b108eff1bfca9469c3fcd7e18e014676d05cf81acdd2bc0"}
Aug 04 18:01:02.890225 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.890196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jcl8j" event={"ID":"43e87821-b4a3-4117-84cd-3f5410422a03","Type":"ContainerStarted","Data":"74238250f688388793f7dc80682258975a334480a83080d3a887eec4f3e3b2b5"}
Aug 04 18:01:02.891604 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.891580    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kkvwm" event={"ID":"5204927c-d0ff-4e9b-830e-f06d22586a17","Type":"ContainerStarted","Data":"561aa5926daabb0bb75887bc45b29dd0c5c646280e4956bde40c265b9560914e"}
Aug 04 18:01:02.892923 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.892906    2562 generic.go:356] "Generic (PLEG): container finished" podID="841bbbda-ca31-4cdf-8374-929647c20747" containerID="25b2223fa64c85e3a602e1dddceb2df1ab84b4a34fdb385c45a18374ac44a363" exitCode=0
Aug 04 18:01:02.893016 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.892964    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerDied","Data":"25b2223fa64c85e3a602e1dddceb2df1ab84b4a34fdb385c45a18374ac44a363"}
Aug 04 18:01:02.894496 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.894474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vx46c" event={"ID":"6885ce7b-c7f5-4abf-8ebf-6a11fa1e55c9","Type":"ContainerStarted","Data":"2b60156030e67f35cf885980559d8d3dcd0d0ba9eea55fd3fc257af711146d07"}
Aug 04 18:01:02.897173 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.897156    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"aadc3cf4c038424deeb732b97c1e187d3f7327b6ba88ab5a5418405f90b9b518"}
Aug 04 18:01:02.897262 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.897211    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"07fe83b89474985994e83b17591400b05e26e7b838052b64f88bc5e5263a2f30"}
Aug 04 18:01:02.897262 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.897242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"3f7a80d0a3693ed8a8248abb4ed514995cc5a59d0b4c72ed38d7e4dd70454f0b"}
Aug 04 18:01:02.897334 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.897269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"7f3e76cb255fd6f05bcd653d2d4cc1b4b4e00f088474a16f0546adb5f75ea285"}
Aug 04 18:01:02.897334 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.897305    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"ed5f857fef8ae5aa1c556b74f0f6a6bd0cb882e6960037351444427fda8e9a21"}
Aug 04 18:01:02.897413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.897335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"86cf292c030c63b649ac688ba638112e7663cf387af7a0f8d083e7c8710c6555"}
Aug 04 18:01:02.905779 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.905741    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x9p9n" podStartSLOduration=4.562546513 podStartE2EDuration="22.905730092s" totalImagesPullingTime="18.343183579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:02.905694716 +0000 UTC m=+22.588143035" watchObservedRunningTime="2026-08-04 18:01:02.905730092 +0000 UTC m=+22.588178410"
Aug 04 18:01:02.906146 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.906120    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-239.ec2.internal" podStartSLOduration=21.90611449 podStartE2EDuration="21.90611449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:00:45.87191626 +0000 UTC m=+5.554364579" watchObservedRunningTime="2026-08-04 18:01:02.90611449 +0000 UTC m=+22.588562810"
Aug 04 18:01:02.950018 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.949980    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vx46c" podStartSLOduration=4.636028573 podStartE2EDuration="22.949969566s" totalImagesPullingTime="18.313940993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:02.949915253 +0000 UTC m=+22.632363573" watchObservedRunningTime="2026-08-04 18:01:02.949969566 +0000 UTC m=+22.632417920"
Aug 04 18:01:02.982895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.982860    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jcl8j" podStartSLOduration=4.673822453 podStartE2EDuration="22.982849812s" totalImagesPullingTime="18.309027359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:02.966140218 +0000 UTC m=+22.648588535" watchObservedRunningTime="2026-08-04 18:01:02.982849812 +0000 UTC m=+22.665298129"
Aug 04 18:01:02.983025 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:02.983008    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vt5k6" podStartSLOduration=4.675278184 podStartE2EDuration="22.983002964s" totalImagesPullingTime="18.30772478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:02.982658241 +0000 UTC m=+22.665106559" watchObservedRunningTime="2026-08-04 18:01:02.983002964 +0000 UTC m=+22.665451281"
Aug 04 18:01:03.001218 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:03.001184    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kkvwm" podStartSLOduration=8.612919446 podStartE2EDuration="23.001173243s" totalImagesPullingTime="14.388253797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:03.000874722 +0000 UTC m=+22.683323039" watchObservedRunningTime="2026-08-04 18:01:03.001173243 +0000 UTC m=+22.683621562"
Aug 04 18:01:03.025075 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:03.025056    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:01:03.025532 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:03.025517    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:01:03.802870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:03.802834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:03.802870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:03.802865    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:03.803306 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:03.802959    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:03.803306 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:03.803089    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:01:03.901318 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:03.901272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mczn8" event={"ID":"e6705ffb-493b-401b-92f7-2ead628737a3","Type":"ContainerStarted","Data":"3bf2150cfb7aa12516c3337bc785a43b2cf22a099f8f0f6a93dee171e6d35c26"}
Aug 04 18:01:03.918663 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:03.918624    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mczn8" podStartSLOduration=9.531882522 podStartE2EDuration="23.91861053s" totalImagesPullingTime="14.386728008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:03.918161248 +0000 UTC m=+23.600609568" watchObservedRunningTime="2026-08-04 18:01:03.91861053 +0000 UTC m=+23.601058847"
Aug 04 18:01:04.611883 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.611857    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 18:01:04.725786 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.725640    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T18:01:04.61187966Z","UUID":"f8ee9ee4-3073-431a-a9ec-57daaca31608","Handler":null,"Name":"","Endpoint":""}
Aug 04 18:01:04.727488 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.727464    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 18:01:04.727621 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.727532    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 18:01:04.802197 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.802171    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:04.802315 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:04.802297    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:04.906213 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.906174    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj" event={"ID":"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d","Type":"ContainerStarted","Data":"cbd53c4512dd863a424c51d85b03b5e0d07d4691198a3e6ec67f4ffdedbe4c91"}
Aug 04 18:01:04.910129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.910109    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 04 18:01:04.910129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:04.910119    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"5777d6a2d32e3b8b173af8e6735b9812e17d1fcbdf5efb35490a785f3b6bfed1"}
Aug 04 18:01:05.371286 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:05.371252    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:01:05.371973 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:05.371947    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jcl8j"
Aug 04 18:01:05.803017 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:05.802989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:05.803196 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:05.803097    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:05.803196 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:05.803152    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:05.803294 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:05.803257    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:01:06.802628 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:06.802599    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:06.802977 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:06.802704    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:07.802342 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.802166    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:07.802476 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.802217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:07.802476 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:07.802410    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:01:07.802561 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:07.802476    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:07.918291 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.918259    2562 generic.go:356] "Generic (PLEG): container finished" podID="841bbbda-ca31-4cdf-8374-929647c20747" containerID="017b93a2a83214131a59ec8d37b4dced9660c8155013fffb016e62724934fffd" exitCode=0
Aug 04 18:01:07.919081 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.918340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerDied","Data":"017b93a2a83214131a59ec8d37b4dced9660c8155013fffb016e62724934fffd"}
Aug 04 18:01:07.922017 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.921993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" event={"ID":"4f1edb22-437e-44e2-96b1-f7de7c538fe6","Type":"ContainerStarted","Data":"6bcfa58d05c626239306c9009eea4490977009d30028f33223b46dff266a0239"}
Aug 04 18:01:07.922338 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.922320    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:01:07.924129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.924109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj" event={"ID":"c4ea25f0-614d-4356-ab3f-e79c4cf3b15d","Type":"ContainerStarted","Data":"d7e59248778eec3876f6055a648ae435c56fd79058e30a166c7b43ba78785b50"}
Aug 04 18:01:07.937641 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.937624    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:01:07.966438 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.966400    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvrwj" podStartSLOduration=4.904786749 podStartE2EDuration="27.966388209s" totalImagesPullingTime="23.06160146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:07.966200489 +0000 UTC m=+27.648648808" watchObservedRunningTime="2026-08-04 18:01:07.966388209 +0000 UTC m=+27.648836561"
Aug 04 18:01:07.995770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:07.995734    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr" podStartSLOduration=9.531839033 podStartE2EDuration="27.995722125s" totalImagesPullingTime="18.463883092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:07.995199555 +0000 UTC m=+27.677647874" watchObservedRunningTime="2026-08-04 18:01:07.995722125 +0000 UTC m=+27.678170443"
Aug 04 18:01:08.803373 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:08.803133    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:08.803552 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:08.803477    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:08.926249 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:08.926225    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:01:08.926626 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:08.926314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:01:08.941515 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:08.941468    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:01:09.494208 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:09.494132    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gw46q"]
Aug 04 18:01:09.494320 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:09.494277    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:09.494384 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:09.494365    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:01:09.496145 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:09.496118    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2qbzg"]
Aug 04 18:01:09.496258 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:09.496215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:09.496307 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:09.496285    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:09.496621 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:09.496602    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rvmpt"]
Aug 04 18:01:09.496713 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:09.496676    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:09.496781 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:09.496764    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:10.803107 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:10.803077    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:10.803454 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:10.803169    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:01:10.803454 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:10.803241    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:10.803454 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:10.803325    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:10.933190 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:10.933159    2562 generic.go:356] "Generic (PLEG): container finished" podID="841bbbda-ca31-4cdf-8374-929647c20747" containerID="12928ab2e3118f07ed0d412b64403636a386ea4afd13f4be76dd9b992641246a" exitCode=0
Aug 04 18:01:10.933284 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:10.933240    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerDied","Data":"12928ab2e3118f07ed0d412b64403636a386ea4afd13f4be76dd9b992641246a"}
Aug 04 18:01:11.802672 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:11.802492    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:11.802834 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:11.802737    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:12.803157 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:12.803125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:12.803525 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:12.803137    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:12.803525 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:12.803269    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gw46q" podUID="4ce8f519-594e-43eb-9e6e-45143be8e3c8"
Aug 04 18:01:12.803525 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:12.803357    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qbzg" podUID="e5b59001-bd6e-4030-83f7-0d46dcc42501"
Aug 04 18:01:13.803056 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:13.803029    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:13.803226 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:13.803121    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvmpt" podUID="d5691a2d-bea0-40f6-a2d2-6415a59a4c93"
Aug 04 18:01:14.426903 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.426872    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:14.427124 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.426980    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:01:14.427124 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.427028    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs podName:4ce8f519-594e-43eb-9e6e-45143be8e3c8 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:46.427015038 +0000 UTC m=+66.109463337 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs") pod "network-metrics-daemon-gw46q" (UID: "4ce8f519-594e-43eb-9e6e-45143be8e3c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:01:14.528008 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.527985    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:14.528084 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.528023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:14.528151 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.528132    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:01:14.528151 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.528145    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:01:14.528212 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.528162    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:01:14.528212 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.528172    2562 projected.go:196] Error preparing data for projected volume kube-api-access-5gttp for pod openshift-network-diagnostics/network-check-target-rvmpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:01:14.528212 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.528195    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret podName:e5b59001-bd6e-4030-83f7-0d46dcc42501 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:46.528177717 +0000 UTC m=+66.210626013 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret") pod "global-pull-secret-syncer-2qbzg" (UID: "e5b59001-bd6e-4030-83f7-0d46dcc42501") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:01:14.528313 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:14.528213    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp podName:d5691a2d-bea0-40f6-a2d2-6415a59a4c93 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:46.528200846 +0000 UTC m=+66.210649142 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gttp" (UniqueName: "kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp") pod "network-check-target-rvmpt" (UID: "d5691a2d-bea0-40f6-a2d2-6415a59a4c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:01:14.679497 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.679435    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-239.ec2.internal" event="NodeReady"
Aug 04 18:01:14.679635 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.679567    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 18:01:14.726888 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.726864    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v8j5g"]
Aug 04 18:01:14.758388 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.758361    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5vwvz"]
Aug 04 18:01:14.758513 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.758483    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:14.761325 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.761301    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 18:01:14.761468 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.761344    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 18:01:14.761468 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.761354    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 18:01:14.761589 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.761558    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rwpcl\""
Aug 04 18:01:14.771650 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.771631    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v8j5g"]
Aug 04 18:01:14.771740 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.771659    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5vwvz"]
Aug 04 18:01:14.771740 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.771735    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:14.774566 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.774545    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nwtj6\""
Aug 04 18:01:14.774566 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.774560    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 18:01:14.774706 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.774567    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 18:01:14.802677 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.802657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:14.802765 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.802695    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:14.805289 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.805274    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 18:01:14.805628 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.805410    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bbhlr\""
Aug 04 18:01:14.805628 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.805544    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 18:01:14.930178 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.930127    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:14.930269 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.930177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-config-volume\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:14.930269 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.930207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxt5w\" (UniqueName: \"kubernetes.io/projected/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-kube-api-access-bxt5w\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:14.930269 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.930225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rggl\" (UniqueName: \"kubernetes.io/projected/52b51669-cef4-45e0-bc42-a96cc97f27e8-kube-api-access-7rggl\") pod \"ingress-canary-v8j5g\" (UID: \"52b51669-cef4-45e0-bc42-a96cc97f27e8\") " pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:14.930269 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.930255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b51669-cef4-45e0-bc42-a96cc97f27e8-cert\") pod \"ingress-canary-v8j5g\" (UID: \"52b51669-cef4-45e0-bc42-a96cc97f27e8\") " pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:14.930395 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:14.930310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-tmp-dir\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.030600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.030576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.030716 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.030641    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-config-volume\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.040576 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.040556    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxt5w\" (UniqueName: \"kubernetes.io/projected/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-kube-api-access-bxt5w\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.040576 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.040584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rggl\" (UniqueName: \"kubernetes.io/projected/52b51669-cef4-45e0-bc42-a96cc97f27e8-kube-api-access-7rggl\") pod \"ingress-canary-v8j5g\" (UID: \"52b51669-cef4-45e0-bc42-a96cc97f27e8\") " pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:15.040759 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.040635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b51669-cef4-45e0-bc42-a96cc97f27e8-cert\") pod \"ingress-canary-v8j5g\" (UID: \"52b51669-cef4-45e0-bc42-a96cc97f27e8\") " pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:15.040759 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.040660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-tmp-dir\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.040848 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:15.040767    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:01:15.040848 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:15.040827    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls podName:4a651095-5402-42ec-aaae-6bbbfe3f7cf3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:15.540808319 +0000 UTC m=+35.223256620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls") pod "dns-default-5vwvz" (UID: "4a651095-5402-42ec-aaae-6bbbfe3f7cf3") : secret "dns-default-metrics-tls" not found
Aug 04 18:01:15.041134 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.041114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-tmp-dir\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.041216 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.041151    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-config-volume\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.045300 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.045278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b51669-cef4-45e0-bc42-a96cc97f27e8-cert\") pod \"ingress-canary-v8j5g\" (UID: \"52b51669-cef4-45e0-bc42-a96cc97f27e8\") " pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:15.053044 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.053021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rggl\" (UniqueName: \"kubernetes.io/projected/52b51669-cef4-45e0-bc42-a96cc97f27e8-kube-api-access-7rggl\") pod \"ingress-canary-v8j5g\" (UID: \"52b51669-cef4-45e0-bc42-a96cc97f27e8\") " pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:15.053145 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.053057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxt5w\" (UniqueName: \"kubernetes.io/projected/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-kube-api-access-bxt5w\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.067914 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.067895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v8j5g"
Aug 04 18:01:15.235534 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.235488    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v8j5g"]
Aug 04 18:01:15.271617 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:15.271587    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b51669_cef4_45e0_bc42_a96cc97f27e8.slice/crio-8ddfb14827c062385351d00eb1f195d081273d344fb0db1a1383bb481c4da855 WatchSource:0}: Error finding container 8ddfb14827c062385351d00eb1f195d081273d344fb0db1a1383bb481c4da855: Status 404 returned error can't find the container with id 8ddfb14827c062385351d00eb1f195d081273d344fb0db1a1383bb481c4da855
Aug 04 18:01:15.544258 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.544234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:15.544406 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:15.544386    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:01:15.544453 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:15.544447    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls podName:4a651095-5402-42ec-aaae-6bbbfe3f7cf3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:16.544429741 +0000 UTC m=+36.226878046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls") pod "dns-default-5vwvz" (UID: "4a651095-5402-42ec-aaae-6bbbfe3f7cf3") : secret "dns-default-metrics-tls" not found
Aug 04 18:01:15.802239 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.802216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:15.805306 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.805286    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 18:01:15.806417 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.806397    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 18:01:15.806417 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.806410    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x566h\""
Aug 04 18:01:15.945214 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.945179    2562 generic.go:356] "Generic (PLEG): container finished" podID="841bbbda-ca31-4cdf-8374-929647c20747" containerID="014351a6573591178f601633ab78574691fcb734ba9a7087df2593d004ed482f" exitCode=0
Aug 04 18:01:15.945390 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.945259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerDied","Data":"014351a6573591178f601633ab78574691fcb734ba9a7087df2593d004ed482f"}
Aug 04 18:01:15.946420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:15.946391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v8j5g" event={"ID":"52b51669-cef4-45e0-bc42-a96cc97f27e8","Type":"ContainerStarted","Data":"8ddfb14827c062385351d00eb1f195d081273d344fb0db1a1383bb481c4da855"}
Aug 04 18:01:16.550346 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:16.550317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:16.550497 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:16.550455    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:01:16.550563 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:16.550529    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls podName:4a651095-5402-42ec-aaae-6bbbfe3f7cf3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:18.550514759 +0000 UTC m=+38.232963070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls") pod "dns-default-5vwvz" (UID: "4a651095-5402-42ec-aaae-6bbbfe3f7cf3") : secret "dns-default-metrics-tls" not found
Aug 04 18:01:17.626590 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.626546    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"]
Aug 04 18:01:17.650343 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.650319    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"]
Aug 04 18:01:17.650492 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.650356    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"]
Aug 04 18:01:17.650492 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.650457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.653084 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.653062    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Aug 04 18:01:17.653463 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.653444    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Aug 04 18:01:17.653605 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.653472    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:01:17.653605 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.653579    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Aug 04 18:01:17.653605 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.653472    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-bgxgp\""
Aug 04 18:01:17.658193 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.658173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-serving-cert\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.658323 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.658198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xftqq\" (UniqueName: \"kubernetes.io/projected/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-kube-api-access-xftqq\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.658323 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.658250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-config\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.666431 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.666401    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"]
Aug 04 18:01:17.666577 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.666561    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.669939 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.669717    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Aug 04 18:01:17.669939 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.669743    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-b2s5g\""
Aug 04 18:01:17.669939 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.669754    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:01:17.669939 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.669870    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Aug 04 18:01:17.670393 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.670373    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Aug 04 18:01:17.684590 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.684571    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"]
Aug 04 18:01:17.684719 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.684703    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"
Aug 04 18:01:17.687600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.687527    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Aug 04 18:01:17.687600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.687527    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:01:17.687756 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.687603    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bdvds\""
Aug 04 18:01:17.703580 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.703557    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"]
Aug 04 18:01:17.703659 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.703585    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"]
Aug 04 18:01:17.703659 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.703608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"]
Aug 04 18:01:17.703744 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.703672    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:17.706272 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.706226    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Aug 04 18:01:17.706393 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.706279    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:01:17.706837 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.706819    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Aug 04 18:01:17.707051 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.707032    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bb5qp\""
Aug 04 18:01:17.722984 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.722962    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"]
Aug 04 18:01:17.737776 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.737754    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"]
Aug 04 18:01:17.737881 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.737854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"
Aug 04 18:01:17.740856 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.740834    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-p29kk\""
Aug 04 18:01:17.759093 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/386c9d7a-67a4-432d-b226-97305d53924a-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.759219 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dtrr\" (UniqueName: \"kubernetes.io/projected/62793eed-18b9-4c4f-82d9-d75ac2559949-kube-api-access-6dtrr\") pod \"volume-data-source-validator-5b8776669-zk68b\" (UID: \"62793eed-18b9-4c4f-82d9-d75ac2559949\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"
Aug 04 18:01:17.759285 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-serving-cert\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.759340 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftqq\" (UniqueName: \"kubernetes.io/projected/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-kube-api-access-xftqq\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.759340 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759329    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:17.759447 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dtln\" (UniqueName: \"kubernetes.io/projected/043fd442-0b84-41aa-a152-65ec84d2efa3-kube-api-access-2dtln\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:17.759447 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759398    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/386c9d7a-67a4-432d-b226-97305d53924a-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.759447 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-config\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.759664 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgf2x\" (UniqueName: \"kubernetes.io/projected/dcfa6f11-3782-4295-bfa7-b3bfa4c61403-kube-api-access-xgf2x\") pod \"network-check-source-65bdbbff45-mrq5b\" (UID: \"dcfa6f11-3782-4295-bfa7-b3bfa4c61403\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"
Aug 04 18:01:17.759664 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.759514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5gp7\" (UniqueName: \"kubernetes.io/projected/386c9d7a-67a4-432d-b226-97305d53924a-kube-api-access-l5gp7\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.762254 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.762235    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-config\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.763384 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.763359    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-serving-cert\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.772902 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.772876    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftqq\" (UniqueName: \"kubernetes.io/projected/fd44f2d2-2f07-492e-b98f-a0cc23ab3873-kube-api-access-xftqq\") pod \"service-ca-operator-784f97bcf6-4q8wn\" (UID: \"fd44f2d2-2f07-492e-b98f-a0cc23ab3873\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.829041 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.829008    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86bd98686-jnm59"]
Aug 04 18:01:17.839242 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.839218    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.842379 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.842354    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 18:01:17.842486 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.842368    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9xbrz\""
Aug 04 18:01:17.842486 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.842362    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 18:01:17.842657 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.842607    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 18:01:17.851174 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.851153    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86bd98686-jnm59"]
Aug 04 18:01:17.856863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.856840    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 18:01:17.859817 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.859799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-installation-pull-secrets\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.859916 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.859824    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-bound-sa-token\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.859974 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.859929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:17.859974 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.859960    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7ea64979-84ea-4b18-abb4-3b5e5adb5670-ca-trust-extracted\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.860059 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.859980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dtln\" (UniqueName: \"kubernetes.io/projected/043fd442-0b84-41aa-a152-65ec84d2efa3-kube-api-access-2dtln\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:17.860059 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/386c9d7a-67a4-432d-b226-97305d53924a-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.860059 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:17.860025    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 18:01:17.860193 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:17.860088    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls podName:043fd442-0b84-41aa-a152-65ec84d2efa3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:18.360067285 +0000 UTC m=+38.042515581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-j4r5c" (UID: "043fd442-0b84-41aa-a152-65ec84d2efa3") : secret "samples-operator-tls" not found
Aug 04 18:01:17.860193 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-image-registry-private-configuration\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.860193 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-certificates\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.860349 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgf2x\" (UniqueName: \"kubernetes.io/projected/dcfa6f11-3782-4295-bfa7-b3bfa4c61403-kube-api-access-xgf2x\") pod \"network-check-source-65bdbbff45-mrq5b\" (UID: \"dcfa6f11-3782-4295-bfa7-b3bfa4c61403\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"
Aug 04 18:01:17.860349 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5gp7\" (UniqueName: \"kubernetes.io/projected/386c9d7a-67a4-432d-b226-97305d53924a-kube-api-access-l5gp7\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.860349 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.860349 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvlgq\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-kube-api-access-jvlgq\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.860349 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/386c9d7a-67a4-432d-b226-97305d53924a-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.860349 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-trusted-ca\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.860640 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dtrr\" (UniqueName: \"kubernetes.io/projected/62793eed-18b9-4c4f-82d9-d75ac2559949-kube-api-access-6dtrr\") pod \"volume-data-source-validator-5b8776669-zk68b\" (UID: \"62793eed-18b9-4c4f-82d9-d75ac2559949\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"
Aug 04 18:01:17.860640 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.860617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/386c9d7a-67a4-432d-b226-97305d53924a-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.862773 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.862754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/386c9d7a-67a4-432d-b226-97305d53924a-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.875950 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.875905    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dtln\" (UniqueName: \"kubernetes.io/projected/043fd442-0b84-41aa-a152-65ec84d2efa3-kube-api-access-2dtln\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:17.880914 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.880866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5gp7\" (UniqueName: \"kubernetes.io/projected/386c9d7a-67a4-432d-b226-97305d53924a-kube-api-access-l5gp7\") pod \"kube-storage-version-migrator-operator-7878f4b88c-xjm4d\" (UID: \"386c9d7a-67a4-432d-b226-97305d53924a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.882073 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.882052    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dtrr\" (UniqueName: \"kubernetes.io/projected/62793eed-18b9-4c4f-82d9-d75ac2559949-kube-api-access-6dtrr\") pod \"volume-data-source-validator-5b8776669-zk68b\" (UID: \"62793eed-18b9-4c4f-82d9-d75ac2559949\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"
Aug 04 18:01:17.885014 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.884991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgf2x\" (UniqueName: \"kubernetes.io/projected/dcfa6f11-3782-4295-bfa7-b3bfa4c61403-kube-api-access-xgf2x\") pod \"network-check-source-65bdbbff45-mrq5b\" (UID: \"dcfa6f11-3782-4295-bfa7-b3bfa4c61403\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"
Aug 04 18:01:17.960814 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.960759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-trusted-ca\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961060 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.960854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-installation-pull-secrets\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961060 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.960882    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-bound-sa-token\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961060 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.960952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7ea64979-84ea-4b18-abb4-3b5e5adb5670-ca-trust-extracted\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961060 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.960986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-image-registry-private-configuration\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961060 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.961011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-certificates\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961060 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.961037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961060 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.961059    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvlgq\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-kube-api-access-jvlgq\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961490 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.961365    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"
Aug 04 18:01:17.961662 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:17.961640    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:01:17.961726 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:17.961665    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bd98686-jnm59: secret "image-registry-tls" not found
Aug 04 18:01:17.961782 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.961733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7ea64979-84ea-4b18-abb4-3b5e5adb5670-ca-trust-extracted\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961782 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:17.961743    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls podName:7ea64979-84ea-4b18-abb4-3b5e5adb5670 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:18.461722373 +0000 UTC m=+38.144170683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls") pod "image-registry-86bd98686-jnm59" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670") : secret "image-registry-tls" not found
Aug 04 18:01:17.961891 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.961785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-trusted-ca\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.961950 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.961886    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-certificates\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.964699 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.964671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-installation-pull-secrets\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.964787 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.964671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-image-registry-private-configuration\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.972038 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.972014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-bound-sa-token\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.972139 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.972120    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvlgq\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-kube-api-access-jvlgq\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:17.978955 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.978917    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"
Aug 04 18:01:17.994602 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:17.994578    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"
Aug 04 18:01:18.050981 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.050876    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"
Aug 04 18:01:18.157521 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.154256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn"]
Aug 04 18:01:18.168541 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:18.166404    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd44f2d2_2f07_492e_b98f_a0cc23ab3873.slice/crio-6d75993fe66cc19eba9015af2ae89ad7be83f6cae962b5fd6024f719d46e7f84 WatchSource:0}: Error finding container 6d75993fe66cc19eba9015af2ae89ad7be83f6cae962b5fd6024f719d46e7f84: Status 404 returned error can't find the container with id 6d75993fe66cc19eba9015af2ae89ad7be83f6cae962b5fd6024f719d46e7f84
Aug 04 18:01:18.173089 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.172129    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b"]
Aug 04 18:01:18.182369 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:18.182334    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62793eed_18b9_4c4f_82d9_d75ac2559949.slice/crio-479525e8a260da6b56298e7512023e151e216dd799e086ead37a5e7ed66d1eae WatchSource:0}: Error finding container 479525e8a260da6b56298e7512023e151e216dd799e086ead37a5e7ed66d1eae: Status 404 returned error can't find the container with id 479525e8a260da6b56298e7512023e151e216dd799e086ead37a5e7ed66d1eae
Aug 04 18:01:18.191856 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.191832    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d"]
Aug 04 18:01:18.197072 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:18.197045    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod386c9d7a_67a4_432d_b226_97305d53924a.slice/crio-3448879a8c2e897a0f2058c60dbae59e9e0b08f3b5bed1a7f134cd97344b43dd WatchSource:0}: Error finding container 3448879a8c2e897a0f2058c60dbae59e9e0b08f3b5bed1a7f134cd97344b43dd: Status 404 returned error can't find the container with id 3448879a8c2e897a0f2058c60dbae59e9e0b08f3b5bed1a7f134cd97344b43dd
Aug 04 18:01:18.246521 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.246481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b"]
Aug 04 18:01:18.250838 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:18.250798    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfa6f11_3782_4295_bfa7_b3bfa4c61403.slice/crio-38f82311044be24bb6bf78623dbe8d4d559362da894d5a96678ad7e64ba69d33 WatchSource:0}: Error finding container 38f82311044be24bb6bf78623dbe8d4d559362da894d5a96678ad7e64ba69d33: Status 404 returned error can't find the container with id 38f82311044be24bb6bf78623dbe8d4d559362da894d5a96678ad7e64ba69d33
Aug 04 18:01:18.363879 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.363843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:18.364045 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:18.364022    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 18:01:18.364117 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:18.364099    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls podName:043fd442-0b84-41aa-a152-65ec84d2efa3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:19.364076293 +0000 UTC m=+39.046524588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-j4r5c" (UID: "043fd442-0b84-41aa-a152-65ec84d2efa3") : secret "samples-operator-tls" not found
Aug 04 18:01:18.464661 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.464521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:18.464829 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:18.464717    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:01:18.464829 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:18.464739    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bd98686-jnm59: secret "image-registry-tls" not found
Aug 04 18:01:18.464829 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:18.464807    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls podName:7ea64979-84ea-4b18-abb4-3b5e5adb5670 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:19.46478689 +0000 UTC m=+39.147235189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls") pod "image-registry-86bd98686-jnm59" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670") : secret "image-registry-tls" not found
Aug 04 18:01:18.565653 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.565621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:18.565832 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:18.565797    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:01:18.565893 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:18.565876    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls podName:4a651095-5402-42ec-aaae-6bbbfe3f7cf3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:22.565853816 +0000 UTC m=+42.248302116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls") pod "dns-default-5vwvz" (UID: "4a651095-5402-42ec-aaae-6bbbfe3f7cf3") : secret "dns-default-metrics-tls" not found
Aug 04 18:01:18.955844 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.955613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b" event={"ID":"dcfa6f11-3782-4295-bfa7-b3bfa4c61403","Type":"ContainerStarted","Data":"38f82311044be24bb6bf78623dbe8d4d559362da894d5a96678ad7e64ba69d33"}
Aug 04 18:01:18.957264 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.957198    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d" event={"ID":"386c9d7a-67a4-432d-b226-97305d53924a","Type":"ContainerStarted","Data":"3448879a8c2e897a0f2058c60dbae59e9e0b08f3b5bed1a7f134cd97344b43dd"}
Aug 04 18:01:18.958737 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.958681    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b" event={"ID":"62793eed-18b9-4c4f-82d9-d75ac2559949","Type":"ContainerStarted","Data":"479525e8a260da6b56298e7512023e151e216dd799e086ead37a5e7ed66d1eae"}
Aug 04 18:01:18.960657 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:18.960591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn" event={"ID":"fd44f2d2-2f07-492e-b98f-a0cc23ab3873","Type":"ContainerStarted","Data":"6d75993fe66cc19eba9015af2ae89ad7be83f6cae962b5fd6024f719d46e7f84"}
Aug 04 18:01:19.373404 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:19.373317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:19.373622 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:19.373449    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 18:01:19.373622 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:19.373550    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls podName:043fd442-0b84-41aa-a152-65ec84d2efa3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:21.373528395 +0000 UTC m=+41.055976693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-j4r5c" (UID: "043fd442-0b84-41aa-a152-65ec84d2efa3") : secret "samples-operator-tls" not found
Aug 04 18:01:19.474187 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:19.474003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:19.474187 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:19.474111    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:01:19.474187 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:19.474129    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bd98686-jnm59: secret "image-registry-tls" not found
Aug 04 18:01:19.474187 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:19.474193    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls podName:7ea64979-84ea-4b18-abb4-3b5e5adb5670 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:21.474174275 +0000 UTC m=+41.156622574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls") pod "image-registry-86bd98686-jnm59" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670") : secret "image-registry-tls" not found
Aug 04 18:01:21.387231 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:21.387177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:21.387729 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:21.387349    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 18:01:21.387729 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:21.387429    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls podName:043fd442-0b84-41aa-a152-65ec84d2efa3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:25.38740693 +0000 UTC m=+45.069855226 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-j4r5c" (UID: "043fd442-0b84-41aa-a152-65ec84d2efa3") : secret "samples-operator-tls" not found
Aug 04 18:01:21.487711 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:21.487668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:21.487899 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:21.487886    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:01:21.487958 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:21.487901    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bd98686-jnm59: secret "image-registry-tls" not found
Aug 04 18:01:21.488033 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:21.487963    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls podName:7ea64979-84ea-4b18-abb4-3b5e5adb5670 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:25.487944659 +0000 UTC m=+45.170392979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls") pod "image-registry-86bd98686-jnm59" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670") : secret "image-registry-tls" not found
Aug 04 18:01:22.595750 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:22.595718    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:22.596079 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:22.595866    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:01:22.596079 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:22.595925    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls podName:4a651095-5402-42ec-aaae-6bbbfe3f7cf3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:30.595909007 +0000 UTC m=+50.278357307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls") pod "dns-default-5vwvz" (UID: "4a651095-5402-42ec-aaae-6bbbfe3f7cf3") : secret "dns-default-metrics-tls" not found
Aug 04 18:01:23.031080 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.031045    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jlxm4"]
Aug 04 18:01:23.051673 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.051330    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.053431 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.053369    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jlxm4"]
Aug 04 18:01:23.056708 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.056677    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 18:01:23.056835 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.056747    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 18:01:23.056897 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.056874    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 18:01:23.057129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.056966    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 18:01:23.058541 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.058354    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mcf9m\""
Aug 04 18:01:23.101423 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.100124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/664dc729-b7a8-4f4d-9fd6-fbc54f752945-crio-socket\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.101423 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.100168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn7hg\" (UniqueName: \"kubernetes.io/projected/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-api-access-hn7hg\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.101423 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.100338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.101423 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.100390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/664dc729-b7a8-4f4d-9fd6-fbc54f752945-data-volume\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.101423 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.100415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.201371 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.201337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.201555 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.201390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/664dc729-b7a8-4f4d-9fd6-fbc54f752945-data-volume\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.201555 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.201419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.201555 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.201463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/664dc729-b7a8-4f4d-9fd6-fbc54f752945-crio-socket\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.201555 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.201491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn7hg\" (UniqueName: \"kubernetes.io/projected/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-api-access-hn7hg\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.201555 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.201540    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:23.201762 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.201628    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:23.701607355 +0000 UTC m=+43.384055656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:23.201823 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.201780    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:01:23.201862 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.201849    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:23.701832705 +0000 UTC m=+43.384281018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : configmap "kube-rbac-proxy" not found
Aug 04 18:01:23.201958 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.201938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/664dc729-b7a8-4f4d-9fd6-fbc54f752945-crio-socket\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.202154 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.202136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/664dc729-b7a8-4f4d-9fd6-fbc54f752945-data-volume\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.214333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.214311    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn7hg\" (UniqueName: \"kubernetes.io/projected/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-api-access-hn7hg\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.706435 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.706399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.706868 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.706458    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:23.706868 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.706578    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:01:23.706868 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.706580    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:23.706868 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.706647    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:24.70662802 +0000 UTC m=+44.389076321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : configmap "kube-rbac-proxy" not found
Aug 04 18:01:23.706868 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:23.706666    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:24.706655411 +0000 UTC m=+44.389103713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:23.978452 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.978375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerStarted","Data":"a84dd4f2d6473a314c86f0c8e847fc939762c652231f1c5dbca826a159fe4872"}
Aug 04 18:01:23.980110 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.980080    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d" event={"ID":"386c9d7a-67a4-432d-b226-97305d53924a","Type":"ContainerStarted","Data":"edfbf0465672bdb5ec66f23690b268a7c79870ed86203f2917db68f3a02dd3a4"}
Aug 04 18:01:23.981600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:23.981572    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v8j5g" event={"ID":"52b51669-cef4-45e0-bc42-a96cc97f27e8","Type":"ContainerStarted","Data":"e2ea9e2aa3aa72142c6f9d6c6180e474789d70647c0b6a1262f440a9f8db9e74"}
Aug 04 18:01:24.032291 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:24.032241    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v8j5g" podStartSLOduration=2.054428995 podStartE2EDuration="10.032226544s" totalImagesPullingTime="7.977797549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:24.030887317 +0000 UTC m=+43.713335634" watchObservedRunningTime="2026-08-04 18:01:24.032226544 +0000 UTC m=+43.714674861"
Aug 04 18:01:24.048814 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:24.048765    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d" podStartSLOduration=1.995634453 podStartE2EDuration="7.048746611s" totalImagesPullingTime="5.053112158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:24.047161663 +0000 UTC m=+43.729609982" watchObservedRunningTime="2026-08-04 18:01:24.048746611 +0000 UTC m=+43.731194931"
Aug 04 18:01:24.482158 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:24.482130    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vt5k6_b4681ca7-7610-4a70-951f-850e67bb9c1d/dns-node-resolver/0.log"
Aug 04 18:01:24.715084 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:24.715053    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:24.715469 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:24.715158    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:24.715469 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:24.715168    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:01:24.715469 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:24.715226    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:26.715210629 +0000 UTC m=+46.397658925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : configmap "kube-rbac-proxy" not found
Aug 04 18:01:24.715469 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:24.715252    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:24.715469 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:24.715293    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:26.715282378 +0000 UTC m=+46.397730674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:24.987838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:24.987792    2562 generic.go:356] "Generic (PLEG): container finished" podID="841bbbda-ca31-4cdf-8374-929647c20747" containerID="a84dd4f2d6473a314c86f0c8e847fc939762c652231f1c5dbca826a159fe4872" exitCode=0
Aug 04 18:01:24.988024 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:24.987890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerDied","Data":"a84dd4f2d6473a314c86f0c8e847fc939762c652231f1c5dbca826a159fe4872"}
Aug 04 18:01:25.422084 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.422051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:25.422225 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:25.422205    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 18:01:25.422282 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:25.422272    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls podName:043fd442-0b84-41aa-a152-65ec84d2efa3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:33.42225573 +0000 UTC m=+53.104704028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-j4r5c" (UID: "043fd442-0b84-41aa-a152-65ec84d2efa3") : secret "samples-operator-tls" not found
Aug 04 18:01:25.522943 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.522910    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:25.523082 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:25.523048    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:01:25.523082 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:25.523066    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bd98686-jnm59: secret "image-registry-tls" not found
Aug 04 18:01:25.523154 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:25.523133    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls podName:7ea64979-84ea-4b18-abb4-3b5e5adb5670 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:33.523119208 +0000 UTC m=+53.205567504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls") pod "image-registry-86bd98686-jnm59" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670") : secret "image-registry-tls" not found
Aug 04 18:01:25.679528 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.679490    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kkvwm_5204927c-d0ff-4e9b-830e-f06d22586a17/node-ca/0.log"
Aug 04 18:01:25.993560 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.993468    2562 generic.go:356] "Generic (PLEG): container finished" podID="841bbbda-ca31-4cdf-8374-929647c20747" containerID="90226e377a5d2896bbb15ffbb40a9172448c4615bd29d064f62a6150b0507160" exitCode=0
Aug 04 18:01:25.994061 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.993555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerDied","Data":"90226e377a5d2896bbb15ffbb40a9172448c4615bd29d064f62a6150b0507160"}
Aug 04 18:01:25.995201 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.995177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b" event={"ID":"62793eed-18b9-4c4f-82d9-d75ac2559949","Type":"ContainerStarted","Data":"0cda67d4a1c144471fd8ca913020dd3eeaddd698343cdb8797c76374360476af"}
Aug 04 18:01:25.996646 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.996623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn" event={"ID":"fd44f2d2-2f07-492e-b98f-a0cc23ab3873","Type":"ContainerStarted","Data":"ee1a99a39557cd44a00fe412814bd47c48e98a3adb3a5c361d7ea4d25905541f"}
Aug 04 18:01:25.998151 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:25.998132    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b" event={"ID":"dcfa6f11-3782-4295-bfa7-b3bfa4c61403","Type":"ContainerStarted","Data":"3f23d59580cf2934a1b1777187ba07d59344e2cdc7b9da412e953e296409fa8c"}
Aug 04 18:01:26.043782 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.043743    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-zk68b" podStartSLOduration=1.683582852 podStartE2EDuration="9.04372987s" totalImagesPullingTime="7.360147018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:26.041893803 +0000 UTC m=+45.724342123" watchObservedRunningTime="2026-08-04 18:01:26.04372987 +0000 UTC m=+45.726178237"
Aug 04 18:01:26.063121 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.063085    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-mrq5b" podStartSLOduration=1.730015933 podStartE2EDuration="9.063071671s" totalImagesPullingTime="7.333055738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:26.062640243 +0000 UTC m=+45.745088562" watchObservedRunningTime="2026-08-04 18:01:26.063071671 +0000 UTC m=+45.745520017"
Aug 04 18:01:26.093317 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.093280    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn" podStartSLOduration=1.672523321 podStartE2EDuration="9.093267268s" totalImagesPullingTime="7.420743947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:26.089488107 +0000 UTC m=+45.771936424" watchObservedRunningTime="2026-08-04 18:01:26.093267268 +0000 UTC m=+45.775715585"
Aug 04 18:01:26.108436 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.108417    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"]
Aug 04 18:01:26.131696 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.129119    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"]
Aug 04 18:01:26.131696 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.129228    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"
Aug 04 18:01:26.132123 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.132102    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Aug 04 18:01:26.132384 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.132358    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Aug 04 18:01:26.132559 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.132538    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wq25c\""
Aug 04 18:01:26.230529 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.230487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk59x\" (UniqueName: \"kubernetes.io/projected/5ae694f1-2a76-42fe-9c00-deb6cf23e9b6-kube-api-access-vk59x\") pod \"migrator-68bc745d87-k9fd9\" (UID: \"5ae694f1-2a76-42fe-9c00-deb6cf23e9b6\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"
Aug 04 18:01:26.280242 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.280222    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v8j5g_52b51669-cef4-45e0-bc42-a96cc97f27e8/serve-healthcheck-canary/0.log"
Aug 04 18:01:26.331750 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.331722    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk59x\" (UniqueName: \"kubernetes.io/projected/5ae694f1-2a76-42fe-9c00-deb6cf23e9b6-kube-api-access-vk59x\") pod \"migrator-68bc745d87-k9fd9\" (UID: \"5ae694f1-2a76-42fe-9c00-deb6cf23e9b6\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"
Aug 04 18:01:26.345011 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.344972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk59x\" (UniqueName: \"kubernetes.io/projected/5ae694f1-2a76-42fe-9c00-deb6cf23e9b6-kube-api-access-vk59x\") pod \"migrator-68bc745d87-k9fd9\" (UID: \"5ae694f1-2a76-42fe-9c00-deb6cf23e9b6\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"
Aug 04 18:01:26.439783 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.439751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"
Aug 04 18:01:26.590086 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.587940    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9"]
Aug 04 18:01:26.591542 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:26.591479    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae694f1_2a76_42fe_9c00_deb6cf23e9b6.slice/crio-6482c7944a17fc210aa88e426b02978cfc0378445ca487130d81a804cabe51cd WatchSource:0}: Error finding container 6482c7944a17fc210aa88e426b02978cfc0378445ca487130d81a804cabe51cd: Status 404 returned error can't find the container with id 6482c7944a17fc210aa88e426b02978cfc0378445ca487130d81a804cabe51cd
Aug 04 18:01:26.735078 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.735046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:26.735244 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:26.735091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:26.735244 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:26.735196    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:26.735244 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:26.735212    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:01:26.735351 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:26.735263    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:30.735247835 +0000 UTC m=+50.417696131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:26.735351 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:26.735279    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:30.735273482 +0000 UTC m=+50.417721778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : configmap "kube-rbac-proxy" not found
Aug 04 18:01:27.001559 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:27.001517    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9" event={"ID":"5ae694f1-2a76-42fe-9c00-deb6cf23e9b6","Type":"ContainerStarted","Data":"6482c7944a17fc210aa88e426b02978cfc0378445ca487130d81a804cabe51cd"}
Aug 04 18:01:27.004946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:27.004913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw958" event={"ID":"841bbbda-ca31-4cdf-8374-929647c20747","Type":"ContainerStarted","Data":"8953a12e8e8eb683747ed1100be022abfedd00f268da5f4e3dc603cef3f539d0"}
Aug 04 18:01:27.031047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:27.031005    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qw958" podStartSLOduration=10.763448431 podStartE2EDuration="47.030991239s" totalImagesPullingTime="36.267542808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:27.029487329 +0000 UTC m=+46.711935646" watchObservedRunningTime="2026-08-04 18:01:27.030991239 +0000 UTC m=+46.713439556"
Aug 04 18:01:29.012131 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.012096    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9" event={"ID":"5ae694f1-2a76-42fe-9c00-deb6cf23e9b6","Type":"ContainerStarted","Data":"4efef7c476cad91051fe8f7633cece41389fdb6bb4e64a0b42ca051909ab10e3"}
Aug 04 18:01:29.012517 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.012145    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9" event={"ID":"5ae694f1-2a76-42fe-9c00-deb6cf23e9b6","Type":"ContainerStarted","Data":"0e1bf14f1574ef81a8aad1db296555b974ec92a2f17d39243ffda018e1d1d077"}
Aug 04 18:01:29.033031 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.032988    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-k9fd9" podStartSLOduration=1.255809124 podStartE2EDuration="3.032973506s" totalImagesPullingTime="1.777164382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:29.032495552 +0000 UTC m=+48.714943870" watchObservedRunningTime="2026-08-04 18:01:29.032973506 +0000 UTC m=+48.715421824"
Aug 04 18:01:29.062054 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.062022    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-nzm4q"]
Aug 04 18:01:29.080440 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.080411    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-nzm4q"]
Aug 04 18:01:29.080589 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.080555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.083333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.083266    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Aug 04 18:01:29.083333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.083309    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Aug 04 18:01:29.083561 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.083349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tvfgc\""
Aug 04 18:01:29.083561 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.083309    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Aug 04 18:01:29.083561 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.083403    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Aug 04 18:01:29.156751 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.156714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz8v6\" (UniqueName: \"kubernetes.io/projected/b8d6743d-c518-49f1-a451-1f7b8c7dd415-kube-api-access-rz8v6\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.156751 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.156748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8d6743d-c518-49f1-a451-1f7b8c7dd415-signing-cabundle\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.156944 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.156879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8d6743d-c518-49f1-a451-1f7b8c7dd415-signing-key\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.258088 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.258050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8d6743d-c518-49f1-a451-1f7b8c7dd415-signing-key\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.258280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.258112    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz8v6\" (UniqueName: \"kubernetes.io/projected/b8d6743d-c518-49f1-a451-1f7b8c7dd415-kube-api-access-rz8v6\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.258280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.258137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8d6743d-c518-49f1-a451-1f7b8c7dd415-signing-cabundle\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.258830 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.258806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8d6743d-c518-49f1-a451-1f7b8c7dd415-signing-cabundle\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.260389 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.260371    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8d6743d-c518-49f1-a451-1f7b8c7dd415-signing-key\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.268143 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.268094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz8v6\" (UniqueName: \"kubernetes.io/projected/b8d6743d-c518-49f1-a451-1f7b8c7dd415-kube-api-access-rz8v6\") pod \"service-ca-98c88fbb5-nzm4q\" (UID: \"b8d6743d-c518-49f1-a451-1f7b8c7dd415\") " pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.389573 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.389534    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q"
Aug 04 18:01:29.533814 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:29.533657    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-nzm4q"]
Aug 04 18:01:29.536059 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:29.536027    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d6743d_c518_49f1_a451_1f7b8c7dd415.slice/crio-eb6ac26f85891e63d2ebf47c6d7fa22579da7cd5fb8661fe677c47a9f334771e WatchSource:0}: Error finding container eb6ac26f85891e63d2ebf47c6d7fa22579da7cd5fb8661fe677c47a9f334771e: Status 404 returned error can't find the container with id eb6ac26f85891e63d2ebf47c6d7fa22579da7cd5fb8661fe677c47a9f334771e
Aug 04 18:01:30.017028 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:30.016998    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q" event={"ID":"b8d6743d-c518-49f1-a451-1f7b8c7dd415","Type":"ContainerStarted","Data":"bfb366951daf156bb0eb51fd756bedfb22223051f1310763dfdb74b82213903a"}
Aug 04 18:01:30.017401 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:30.017047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q" event={"ID":"b8d6743d-c518-49f1-a451-1f7b8c7dd415","Type":"ContainerStarted","Data":"eb6ac26f85891e63d2ebf47c6d7fa22579da7cd5fb8661fe677c47a9f334771e"}
Aug 04 18:01:30.034998 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:30.034924    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-nzm4q" podStartSLOduration=1.034903531 podStartE2EDuration="1.034903531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:30.033363464 +0000 UTC m=+49.715811782" watchObservedRunningTime="2026-08-04 18:01:30.034903531 +0000 UTC m=+49.717351853"
Aug 04 18:01:30.670026 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:30.669988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:30.670203 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:30.670142    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:01:30.670250 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:30.670206    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls podName:4a651095-5402-42ec-aaae-6bbbfe3f7cf3 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:46.670191298 +0000 UTC m=+66.352639594 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls") pod "dns-default-5vwvz" (UID: "4a651095-5402-42ec-aaae-6bbbfe3f7cf3") : secret "dns-default-metrics-tls" not found
Aug 04 18:01:30.770541 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:30.770486    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:30.770717 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:30.770557    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:30.770717 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:30.770632    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:30.770717 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:30.770691    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:38.770677389 +0000 UTC m=+58.453125690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:30.770847 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:30.770631    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:01:30.770847 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:30.770759    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:38.770748108 +0000 UTC m=+58.453196405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : configmap "kube-rbac-proxy" not found
Aug 04 18:01:33.492760 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:33.492713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:33.495080 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:33.495058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/043fd442-0b84-41aa-a152-65ec84d2efa3-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-j4r5c\" (UID: \"043fd442-0b84-41aa-a152-65ec84d2efa3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:33.593603 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:33.593570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:33.593759 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:33.593714    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:01:33.593759 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:33.593743    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bd98686-jnm59: secret "image-registry-tls" not found
Aug 04 18:01:33.593829 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:33.593796    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls podName:7ea64979-84ea-4b18-abb4-3b5e5adb5670 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:49.593781622 +0000 UTC m=+69.276229917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls") pod "image-registry-86bd98686-jnm59" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670") : secret "image-registry-tls" not found
Aug 04 18:01:33.612992 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:33.612972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"
Aug 04 18:01:33.735344 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:33.735303    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c"]
Aug 04 18:01:34.030485 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:34.030409    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c" event={"ID":"043fd442-0b84-41aa-a152-65ec84d2efa3","Type":"ContainerStarted","Data":"33d54451c4a0f457d930bab2e2f097e9ef0ba951a3f8e2ec277d45685bdcfbca"}
Aug 04 18:01:36.038661 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:36.038618    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c" event={"ID":"043fd442-0b84-41aa-a152-65ec84d2efa3","Type":"ContainerStarted","Data":"cdeabdf485242364fd64469b8b35a36acb9d95796e3eb5f2476bf8a948f43a94"}
Aug 04 18:01:36.039034 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:36.038700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c" event={"ID":"043fd442-0b84-41aa-a152-65ec84d2efa3","Type":"ContainerStarted","Data":"eb9d9c89dddd530fce717639ddcadb459ea0f20463601f075b83aa7564171f1e"}
Aug 04 18:01:36.058175 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:36.058135    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-j4r5c" podStartSLOduration=17.548923073 podStartE2EDuration="19.058121507s" totalImagesPullingTime="1.509198434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:36.05680049 +0000 UTC m=+55.739248807" watchObservedRunningTime="2026-08-04 18:01:36.058121507 +0000 UTC m=+55.740569825"
Aug 04 18:01:38.833522 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:38.833473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:38.833899 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:38.833530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:38.833899 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:38.833618    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:38.833899 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:38.833681    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls podName:664dc729-b7a8-4f4d-9fd6-fbc54f752945 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:54.833661745 +0000 UTC m=+74.516110060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jlxm4" (UID: "664dc729-b7a8-4f4d-9fd6-fbc54f752945") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:01:38.834042 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:38.834026    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/664dc729-b7a8-4f4d-9fd6-fbc54f752945-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:40.943477 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:40.943449    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnkpr"
Aug 04 18:01:46.489600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.489560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:46.492675 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.492656    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 18:01:46.502360 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.502339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ce8f519-594e-43eb-9e6e-45143be8e3c8-metrics-certs\") pod \"network-metrics-daemon-gw46q\" (UID: \"4ce8f519-594e-43eb-9e6e-45143be8e3c8\") " pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:46.590760 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.590720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:46.590760 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.590766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:46.593058 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.593038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gttp\" (UniqueName: \"kubernetes.io/projected/d5691a2d-bea0-40f6-a2d2-6415a59a4c93-kube-api-access-5gttp\") pod \"network-check-target-rvmpt\" (UID: \"d5691a2d-bea0-40f6-a2d2-6415a59a4c93\") " pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:46.593628 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.593613    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 18:01:46.603702 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.603684    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e5b59001-bd6e-4030-83f7-0d46dcc42501-original-pull-secret\") pod \"global-pull-secret-syncer-2qbzg\" (UID: \"e5b59001-bd6e-4030-83f7-0d46dcc42501\") " pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:46.614379 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.614361    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bbhlr\""
Aug 04 18:01:46.616486 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.616471    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qbzg"
Aug 04 18:01:46.622585 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.622570    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gw46q"
Aug 04 18:01:46.692268 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.692190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:46.694346 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.694319    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a651095-5402-42ec-aaae-6bbbfe3f7cf3-metrics-tls\") pod \"dns-default-5vwvz\" (UID: \"4a651095-5402-42ec-aaae-6bbbfe3f7cf3\") " pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:46.714117 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.714092    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x566h\""
Aug 04 18:01:46.721306 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.721285    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:46.750187 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.750100    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2qbzg"]
Aug 04 18:01:46.752928 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:46.752896    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b59001_bd6e_4030_83f7_0d46dcc42501.slice/crio-8048067ca26c144f7cab8090bbb1670bee3c0ae941356f12046f2791a6da8879 WatchSource:0}: Error finding container 8048067ca26c144f7cab8090bbb1670bee3c0ae941356f12046f2791a6da8879: Status 404 returned error can't find the container with id 8048067ca26c144f7cab8090bbb1670bee3c0ae941356f12046f2791a6da8879
Aug 04 18:01:46.768221 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.768194    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gw46q"]
Aug 04 18:01:46.772171 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:46.772122    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce8f519_594e_43eb_9e6e_45143be8e3c8.slice/crio-7e659c1a7db31142bf77916bed89d9e9c7dec41aa2e7866183e62ccbd4dcdb05 WatchSource:0}: Error finding container 7e659c1a7db31142bf77916bed89d9e9c7dec41aa2e7866183e62ccbd4dcdb05: Status 404 returned error can't find the container with id 7e659c1a7db31142bf77916bed89d9e9c7dec41aa2e7866183e62ccbd4dcdb05
Aug 04 18:01:46.848933 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.848901    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rvmpt"]
Aug 04 18:01:46.851421 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:46.851397    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5691a2d_bea0_40f6_a2d2_6415a59a4c93.slice/crio-f726d280e62f0c958255b66a21725c724cbeea6fd3fa72bd39dc8da22545073f WatchSource:0}: Error finding container f726d280e62f0c958255b66a21725c724cbeea6fd3fa72bd39dc8da22545073f: Status 404 returned error can't find the container with id f726d280e62f0c958255b66a21725c724cbeea6fd3fa72bd39dc8da22545073f
Aug 04 18:01:46.883113 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.882846    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nwtj6\""
Aug 04 18:01:46.890423 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:46.890386    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:47.015337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.015306    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5vwvz"]
Aug 04 18:01:47.018684 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:47.018656    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a651095_5402_42ec_aaae_6bbbfe3f7cf3.slice/crio-4e05ea88f0a32d380e4551216207251e3df412eef4fe11a9d0563c03a753c151 WatchSource:0}: Error finding container 4e05ea88f0a32d380e4551216207251e3df412eef4fe11a9d0563c03a753c151: Status 404 returned error can't find the container with id 4e05ea88f0a32d380e4551216207251e3df412eef4fe11a9d0563c03a753c151
Aug 04 18:01:47.074576 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.074542    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gw46q" event={"ID":"4ce8f519-594e-43eb-9e6e-45143be8e3c8","Type":"ContainerStarted","Data":"7e659c1a7db31142bf77916bed89d9e9c7dec41aa2e7866183e62ccbd4dcdb05"}
Aug 04 18:01:47.075680 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.075652    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vwvz" event={"ID":"4a651095-5402-42ec-aaae-6bbbfe3f7cf3","Type":"ContainerStarted","Data":"4e05ea88f0a32d380e4551216207251e3df412eef4fe11a9d0563c03a753c151"}
Aug 04 18:01:47.077108 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.077080    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rvmpt" event={"ID":"d5691a2d-bea0-40f6-a2d2-6415a59a4c93","Type":"ContainerStarted","Data":"198a50be92255bf18b755bef1658767fd83436d95c5dd73ba217e091ebdaca02"}
Aug 04 18:01:47.077207 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.077114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rvmpt" event={"ID":"d5691a2d-bea0-40f6-a2d2-6415a59a4c93","Type":"ContainerStarted","Data":"f726d280e62f0c958255b66a21725c724cbeea6fd3fa72bd39dc8da22545073f"}
Aug 04 18:01:47.077207 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.077149    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:01:47.078173 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.078155    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2qbzg" event={"ID":"e5b59001-bd6e-4030-83f7-0d46dcc42501","Type":"ContainerStarted","Data":"8048067ca26c144f7cab8090bbb1670bee3c0ae941356f12046f2791a6da8879"}
Aug 04 18:01:47.102133 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:47.102089    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rvmpt" podStartSLOduration=67.102075102 podStartE2EDuration="1m7.102075102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:47.10078964 +0000 UTC m=+66.783237972" watchObservedRunningTime="2026-08-04 18:01:47.102075102 +0000 UTC m=+66.784523397"
Aug 04 18:01:48.084812 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:48.084774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gw46q" event={"ID":"4ce8f519-594e-43eb-9e6e-45143be8e3c8","Type":"ContainerStarted","Data":"442e885c472eecdcb6086566070d1bceb187c251aa3f61599dfe658c8bb8b285"}
Aug 04 18:01:48.085276 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:48.084840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gw46q" event={"ID":"4ce8f519-594e-43eb-9e6e-45143be8e3c8","Type":"ContainerStarted","Data":"97e36b363670dc4f1df5d9914f566f08ec5c8c5b162466a4c831f90f59108f99"}
Aug 04 18:01:48.105311 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:48.105257    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gw46q" podStartSLOduration=67.222264442 podStartE2EDuration="1m8.105240675s" totalImagesPullingTime="882.976233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:48.104457458 +0000 UTC m=+67.786905778" watchObservedRunningTime="2026-08-04 18:01:48.105240675 +0000 UTC m=+67.787688993"
Aug 04 18:01:49.090894 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.090853    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vwvz" event={"ID":"4a651095-5402-42ec-aaae-6bbbfe3f7cf3","Type":"ContainerStarted","Data":"c2c02f2024ac7e62024c8cd2d642ef727cd2113909ece0a1ba2243bbf552624b"}
Aug 04 18:01:49.091352 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.090914    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vwvz" event={"ID":"4a651095-5402-42ec-aaae-6bbbfe3f7cf3","Type":"ContainerStarted","Data":"72525c01fe5feb82e8a8bf09ee9910e40611aa0eddcf65d77176bed1ba3fadb3"}
Aug 04 18:01:49.091352 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.091077    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:01:49.112250 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.112191    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5vwvz" podStartSLOduration=33.448008226 podStartE2EDuration="35.112170125s" totalImagesPullingTime="1.664161899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:49.110994105 +0000 UTC m=+68.793442424" watchObservedRunningTime="2026-08-04 18:01:49.112170125 +0000 UTC m=+68.794618444"
Aug 04 18:01:49.615248 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.615208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:49.618045 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.618019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"image-registry-86bd98686-jnm59\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") " pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:49.654744 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.654705    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9xbrz\""
Aug 04 18:01:49.661275 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.661236    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:49.814059 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:49.814033    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86bd98686-jnm59"]
Aug 04 18:01:49.816393 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:49.816363    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea64979_84ea_4b18_abb4_3b5e5adb5670.slice/crio-f0b45a10898d57203b647b664dfcfe11934ae3d25047bba87785a8cfe6579e90 WatchSource:0}: Error finding container f0b45a10898d57203b647b664dfcfe11934ae3d25047bba87785a8cfe6579e90: Status 404 returned error can't find the container with id f0b45a10898d57203b647b664dfcfe11934ae3d25047bba87785a8cfe6579e90
Aug 04 18:01:50.097454 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:50.097417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bd98686-jnm59" event={"ID":"7ea64979-84ea-4b18-abb4-3b5e5adb5670","Type":"ContainerStarted","Data":"e45021350d0a190d6926c793f4c454bc3a2c80c7a09f310465bd9ecd4dc2bdad"}
Aug 04 18:01:50.097918 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:50.097476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bd98686-jnm59" event={"ID":"7ea64979-84ea-4b18-abb4-3b5e5adb5670","Type":"ContainerStarted","Data":"f0b45a10898d57203b647b664dfcfe11934ae3d25047bba87785a8cfe6579e90"}
Aug 04 18:01:50.097918 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:50.097763    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:01:50.123389 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:50.123273    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86bd98686-jnm59" podStartSLOduration=33.123255608 podStartE2EDuration="33.123255608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:50.122106928 +0000 UTC m=+69.804555272" watchObservedRunningTime="2026-08-04 18:01:50.123255608 +0000 UTC m=+69.805703925"
Aug 04 18:01:51.656196 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.656152    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"]
Aug 04 18:01:51.660226 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.660188    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"]
Aug 04 18:01:51.660341 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.660319    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.663975 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.663953    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-trnz7\""
Aug 04 18:01:51.664473 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.664452    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 04 18:01:51.664612 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.664541    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 04 18:01:51.664680 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.664657    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 04 18:01:51.664756 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.664731    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 04 18:01:51.664979 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.664957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.667253 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.667236    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Aug 04 18:01:51.670734 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.670712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"]
Aug 04 18:01:51.672716 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.672697    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"]
Aug 04 18:01:51.685910 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.685885    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86bd98686-jnm59"]
Aug 04 18:01:51.734200 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.733117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29f4584a-9df2-4527-bfb8-e600a830b516-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bfc95787d-x5xnm\" (UID: \"29f4584a-9df2-4527-bfb8-e600a830b516\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.734200 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.733178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2kz5\" (UniqueName: \"kubernetes.io/projected/ced1e11e-5bfe-477c-99d1-010cc411ea5e-kube-api-access-r2kz5\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.734200 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.733257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sz86\" (UniqueName: \"kubernetes.io/projected/29f4584a-9df2-4527-bfb8-e600a830b516-kube-api-access-2sz86\") pod \"managed-serviceaccount-addon-agent-6bfc95787d-x5xnm\" (UID: \"29f4584a-9df2-4527-bfb8-e600a830b516\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.734200 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.733311    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ced1e11e-5bfe-477c-99d1-010cc411ea5e-tmp\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.734200 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.733342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ced1e11e-5bfe-477c-99d1-010cc411ea5e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.757401 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.757362    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-97psv"]
Aug 04 18:01:51.760997 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.760967    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-97psv"
Aug 04 18:01:51.761158 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.761043    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55c9667b59-4dzrf"]
Aug 04 18:01:51.764121 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.764099    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.766008 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.765988    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 04 18:01:51.768318 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.768297    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 04 18:01:51.774666 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.774407    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gl79s\""
Aug 04 18:01:51.783891 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.783789    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-97psv"]
Aug 04 18:01:51.788986 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.788959    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55c9667b59-4dzrf"]
Aug 04 18:01:51.834723 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-bound-sa-token\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.834723 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29f4584a-9df2-4527-bfb8-e600a830b516-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bfc95787d-x5xnm\" (UID: \"29f4584a-9df2-4527-bfb8-e600a830b516\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.834937 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2kz5\" (UniqueName: \"kubernetes.io/projected/ced1e11e-5bfe-477c-99d1-010cc411ea5e-kube-api-access-r2kz5\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.834937 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d4pc\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-kube-api-access-8d4pc\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.834937 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jpdb\" (UniqueName: \"kubernetes.io/projected/3c108d6c-e58c-4ce8-9d24-9033bb48d08c-kube-api-access-8jpdb\") pod \"downloads-77c974ffcf-97psv\" (UID: \"3c108d6c-e58c-4ce8-9d24-9033bb48d08c\") " pod="openshift-console/downloads-77c974ffcf-97psv"
Aug 04 18:01:51.834937 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/689ac1de-e938-450c-b0d9-8180e970d9ad-image-registry-private-configuration\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.834937 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/689ac1de-e938-450c-b0d9-8180e970d9ad-installation-pull-secrets\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.835129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sz86\" (UniqueName: \"kubernetes.io/projected/29f4584a-9df2-4527-bfb8-e600a830b516-kube-api-access-2sz86\") pod \"managed-serviceaccount-addon-agent-6bfc95787d-x5xnm\" (UID: \"29f4584a-9df2-4527-bfb8-e600a830b516\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.835129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.834994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/689ac1de-e938-450c-b0d9-8180e970d9ad-ca-trust-extracted\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.835129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.835048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/689ac1de-e938-450c-b0d9-8180e970d9ad-registry-certificates\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.835129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.835079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ced1e11e-5bfe-477c-99d1-010cc411ea5e-tmp\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.835129 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.835100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ced1e11e-5bfe-477c-99d1-010cc411ea5e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.835375 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.835147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/689ac1de-e938-450c-b0d9-8180e970d9ad-trusted-ca\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.835375 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.835201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-registry-tls\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.835490 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.835465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ced1e11e-5bfe-477c-99d1-010cc411ea5e-tmp\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.837336 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.837318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29f4584a-9df2-4527-bfb8-e600a830b516-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bfc95787d-x5xnm\" (UID: \"29f4584a-9df2-4527-bfb8-e600a830b516\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.837425 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.837409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ced1e11e-5bfe-477c-99d1-010cc411ea5e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.851155 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.851125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2kz5\" (UniqueName: \"kubernetes.io/projected/ced1e11e-5bfe-477c-99d1-010cc411ea5e-kube-api-access-r2kz5\") pod \"klusterlet-addon-workmgr-7bc977988b-cvjj7\" (UID: \"ced1e11e-5bfe-477c-99d1-010cc411ea5e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:51.852198 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.852179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sz86\" (UniqueName: \"kubernetes.io/projected/29f4584a-9df2-4527-bfb8-e600a830b516-kube-api-access-2sz86\") pod \"managed-serviceaccount-addon-agent-6bfc95787d-x5xnm\" (UID: \"29f4584a-9df2-4527-bfb8-e600a830b516\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.936607 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/689ac1de-e938-450c-b0d9-8180e970d9ad-trusted-ca\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.936770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-registry-tls\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.936770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-bound-sa-token\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.936770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d4pc\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-kube-api-access-8d4pc\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.936770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jpdb\" (UniqueName: \"kubernetes.io/projected/3c108d6c-e58c-4ce8-9d24-9033bb48d08c-kube-api-access-8jpdb\") pod \"downloads-77c974ffcf-97psv\" (UID: \"3c108d6c-e58c-4ce8-9d24-9033bb48d08c\") " pod="openshift-console/downloads-77c974ffcf-97psv"
Aug 04 18:01:51.936770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/689ac1de-e938-450c-b0d9-8180e970d9ad-image-registry-private-configuration\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.936770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/689ac1de-e938-450c-b0d9-8180e970d9ad-installation-pull-secrets\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.937091 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/689ac1de-e938-450c-b0d9-8180e970d9ad-ca-trust-extracted\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.937091 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.936845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/689ac1de-e938-450c-b0d9-8180e970d9ad-registry-certificates\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.937258 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.937230    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/689ac1de-e938-450c-b0d9-8180e970d9ad-ca-trust-extracted\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.937626 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.937598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/689ac1de-e938-450c-b0d9-8180e970d9ad-trusted-ca\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.937626 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.937617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/689ac1de-e938-450c-b0d9-8180e970d9ad-registry-certificates\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.939356 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.939332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/689ac1de-e938-450c-b0d9-8180e970d9ad-installation-pull-secrets\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.939671 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.939654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/689ac1de-e938-450c-b0d9-8180e970d9ad-image-registry-private-configuration\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.939854 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.939838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-registry-tls\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.951279 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.951252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-bound-sa-token\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.951439 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.951422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d4pc\" (UniqueName: \"kubernetes.io/projected/689ac1de-e938-450c-b0d9-8180e970d9ad-kube-api-access-8d4pc\") pod \"image-registry-55c9667b59-4dzrf\" (UID: \"689ac1de-e938-450c-b0d9-8180e970d9ad\") " pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:51.951652 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.951635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jpdb\" (UniqueName: \"kubernetes.io/projected/3c108d6c-e58c-4ce8-9d24-9033bb48d08c-kube-api-access-8jpdb\") pod \"downloads-77c974ffcf-97psv\" (UID: \"3c108d6c-e58c-4ce8-9d24-9033bb48d08c\") " pod="openshift-console/downloads-77c974ffcf-97psv"
Aug 04 18:01:51.979537 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.979483    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"
Aug 04 18:01:51.987263 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:51.987229    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:52.075926 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.075879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-97psv"
Aug 04 18:01:52.079863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.079704    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:52.109598 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.109100    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2qbzg" event={"ID":"e5b59001-bd6e-4030-83f7-0d46dcc42501","Type":"ContainerStarted","Data":"cb7d1eb1ff8500e93bf9f57c2dda1313711e09a3635d50631d08b1cb5108c09e"}
Aug 04 18:01:52.128764 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.128698    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2qbzg" podStartSLOduration=67.361785107 podStartE2EDuration="1m12.128677179s" totalImagesPullingTime="4.766892072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:52.128348715 +0000 UTC m=+71.810797037" watchObservedRunningTime="2026-08-04 18:01:52.128677179 +0000 UTC m=+71.811125500"
Aug 04 18:01:52.138120 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.137647    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm"]
Aug 04 18:01:52.170950 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.170919    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"]
Aug 04 18:01:52.174278 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:52.174250    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced1e11e_5bfe_477c_99d1_010cc411ea5e.slice/crio-557185e9c691b99b92068475fabf21882878bc68d19dda21efe27ac6334e6c5a WatchSource:0}: Error finding container 557185e9c691b99b92068475fabf21882878bc68d19dda21efe27ac6334e6c5a: Status 404 returned error can't find the container with id 557185e9c691b99b92068475fabf21882878bc68d19dda21efe27ac6334e6c5a
Aug 04 18:01:52.236739 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.236679    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-97psv"]
Aug 04 18:01:52.238954 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:52.238927    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c108d6c_e58c_4ce8_9d24_9033bb48d08c.slice/crio-aa35fbf183bda7390edc74216241dcd40de447f24fa928c2030845c72259f3b9 WatchSource:0}: Error finding container aa35fbf183bda7390edc74216241dcd40de447f24fa928c2030845c72259f3b9: Status 404 returned error can't find the container with id aa35fbf183bda7390edc74216241dcd40de447f24fa928c2030845c72259f3b9
Aug 04 18:01:52.258473 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.258447    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"]
Aug 04 18:01:52.262629 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.262612    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:01:52.266414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.265998    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Aug 04 18:01:52.266414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.266260    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xd9q6\""
Aug 04 18:01:52.266840 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.266769    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55c9667b59-4dzrf"]
Aug 04 18:01:52.269779 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:52.269759    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod689ac1de_e938_450c_b0d9_8180e970d9ad.slice/crio-d302c956a061a0d45f89160ed9fe16241f6debeb89741398affacb25b604882f WatchSource:0}: Error finding container d302c956a061a0d45f89160ed9fe16241f6debeb89741398affacb25b604882f: Status 404 returned error can't find the container with id d302c956a061a0d45f89160ed9fe16241f6debeb89741398affacb25b604882f
Aug 04 18:01:52.273412 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.273386    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"]
Aug 04 18:01:52.339154 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.339122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-dgzgj\" (UID: \"f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:01:52.439770 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.439736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-dgzgj\" (UID: \"f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:01:52.439930 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:52.439868    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Aug 04 18:01:52.439930 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:52.439929    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates podName:f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:52.939911778 +0000 UTC m=+72.622360073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates") pod "prometheus-operator-admission-webhook-66685ff6d4-dgzgj" (UID: "f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06") : secret "prometheus-operator-admission-webhook-tls" not found
Aug 04 18:01:52.943946 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:52.943900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-dgzgj\" (UID: \"f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:01:52.944398 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:52.944051    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Aug 04 18:01:52.944398 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:01:52.944120    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates podName:f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:53.944099732 +0000 UTC m=+73.626548052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates") pod "prometheus-operator-admission-webhook-66685ff6d4-dgzgj" (UID: "f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06") : secret "prometheus-operator-admission-webhook-tls" not found
Aug 04 18:01:53.116400 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.116354    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm" event={"ID":"29f4584a-9df2-4527-bfb8-e600a830b516","Type":"ContainerStarted","Data":"c2a2830d8f3d1f3464606d0046fd7063659f443f0b4b6999a201c6a2bcee2a44"}
Aug 04 18:01:53.117829 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.117788    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-97psv" event={"ID":"3c108d6c-e58c-4ce8-9d24-9033bb48d08c","Type":"ContainerStarted","Data":"aa35fbf183bda7390edc74216241dcd40de447f24fa928c2030845c72259f3b9"}
Aug 04 18:01:53.119276 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.119248    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7" event={"ID":"ced1e11e-5bfe-477c-99d1-010cc411ea5e","Type":"ContainerStarted","Data":"557185e9c691b99b92068475fabf21882878bc68d19dda21efe27ac6334e6c5a"}
Aug 04 18:01:53.121313 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.121283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c9667b59-4dzrf" event={"ID":"689ac1de-e938-450c-b0d9-8180e970d9ad","Type":"ContainerStarted","Data":"c77cb887716b0f66bcfcbfcd61bacaa28416ea280fa0d9e30748d0ae9c11a224"}
Aug 04 18:01:53.121427 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.121342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:01:53.121427 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.121365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c9667b59-4dzrf" event={"ID":"689ac1de-e938-450c-b0d9-8180e970d9ad","Type":"ContainerStarted","Data":"d302c956a061a0d45f89160ed9fe16241f6debeb89741398affacb25b604882f"}
Aug 04 18:01:53.954992 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.954898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-dgzgj\" (UID: \"f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:01:53.962489 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:53.962357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-dgzgj\" (UID: \"f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:01:54.073596 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:54.073392    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:01:54.863233 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:54.863199    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:54.867532 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:54.867483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/664dc729-b7a8-4f4d-9fd6-fbc54f752945-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jlxm4\" (UID: \"664dc729-b7a8-4f4d-9fd6-fbc54f752945\") " pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:55.168158 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:55.167913    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mcf9m\""
Aug 04 18:01:55.175101 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:55.174897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jlxm4"
Aug 04 18:01:57.513247 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:57.513182    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55c9667b59-4dzrf" podStartSLOduration=6.513160251 podStartE2EDuration="6.513160251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:53.153529146 +0000 UTC m=+72.835977470" watchObservedRunningTime="2026-08-04 18:01:57.513160251 +0000 UTC m=+77.195608604"
Aug 04 18:01:57.513863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:57.513845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"]
Aug 04 18:01:57.520552 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:57.520383    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15ed9cc_e46b_4d42_ae94_06f6bc1a9b06.slice/crio-b15df8801cdc4b456551ea32ef95809d3d9fe37719674029dbe20deea861d0cd WatchSource:0}: Error finding container b15df8801cdc4b456551ea32ef95809d3d9fe37719674029dbe20deea861d0cd: Status 404 returned error can't find the container with id b15df8801cdc4b456551ea32ef95809d3d9fe37719674029dbe20deea861d0cd
Aug 04 18:01:57.538359 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:57.538330    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jlxm4"]
Aug 04 18:01:57.541131 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:01:57.540793    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664dc729_b7a8_4f4d_9fd6_fbc54f752945.slice/crio-1190d2a3a5fdb27babf58a55c92f27537ffcbbc8072e3ec9edd8a32f3ac80159 WatchSource:0}: Error finding container 1190d2a3a5fdb27babf58a55c92f27537ffcbbc8072e3ec9edd8a32f3ac80159: Status 404 returned error can't find the container with id 1190d2a3a5fdb27babf58a55c92f27537ffcbbc8072e3ec9edd8a32f3ac80159
Aug 04 18:01:58.147032 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.146990    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm" event={"ID":"29f4584a-9df2-4527-bfb8-e600a830b516","Type":"ContainerStarted","Data":"7ed28b0fd4f007fa56e46973d3da5aa3a2ab822472fb35116552b853a4a3a0a4"}
Aug 04 18:01:58.148686 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.148650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj" event={"ID":"f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06","Type":"ContainerStarted","Data":"b15df8801cdc4b456551ea32ef95809d3d9fe37719674029dbe20deea861d0cd"}
Aug 04 18:01:58.150669 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.150636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7" event={"ID":"ced1e11e-5bfe-477c-99d1-010cc411ea5e","Type":"ContainerStarted","Data":"f24a24bbcfa006667e09447b212d386d44f9fc3c3503f0e3e176494dcf6be6bb"}
Aug 04 18:01:58.151055 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.150872    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:58.152887 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.152865    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7"
Aug 04 18:01:58.153157 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.153123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlxm4" event={"ID":"664dc729-b7a8-4f4d-9fd6-fbc54f752945","Type":"ContainerStarted","Data":"838c2cd208640d549ed032c0588e96d267bf889fc8d620a128e48f8282a6f02f"}
Aug 04 18:01:58.153262 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.153174    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlxm4" event={"ID":"664dc729-b7a8-4f4d-9fd6-fbc54f752945","Type":"ContainerStarted","Data":"1190d2a3a5fdb27babf58a55c92f27537ffcbbc8072e3ec9edd8a32f3ac80159"}
Aug 04 18:01:58.168146 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.168098    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfc95787d-x5xnm" podStartSLOduration=1.9559908670000001 podStartE2EDuration="7.168084159s" totalImagesPullingTime="5.212093292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:58.166065819 +0000 UTC m=+77.848514138" watchObservedRunningTime="2026-08-04 18:01:58.168084159 +0000 UTC m=+77.850532476"
Aug 04 18:01:58.189653 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:58.188987    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bc977988b-cvjj7" podStartSLOduration=1.986134479 podStartE2EDuration="7.188972836s" totalImagesPullingTime="5.202838357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:58.188163373 +0000 UTC m=+77.870611693" watchObservedRunningTime="2026-08-04 18:01:58.188972836 +0000 UTC m=+77.871421155"
Aug 04 18:01:59.099784 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:01:59.099759    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5vwvz"
Aug 04 18:02:00.165031 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:00.164985    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj" event={"ID":"f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06","Type":"ContainerStarted","Data":"59558674849b844837d0a5da046e197de049a6b1de8609c32807c339b66c0da1"}
Aug 04 18:02:00.165483 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:00.165331    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:02:00.171614 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:00.171591    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj"
Aug 04 18:02:00.183098 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:00.183050    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-dgzgj" podStartSLOduration=6.631611095 podStartE2EDuration="8.183037065s" totalImagesPullingTime="1.55142597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:02:00.18246946 +0000 UTC m=+79.864917802" watchObservedRunningTime="2026-08-04 18:02:00.183037065 +0000 UTC m=+79.865485383"
Aug 04 18:02:01.171089 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:01.171048    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlxm4" event={"ID":"664dc729-b7a8-4f4d-9fd6-fbc54f752945","Type":"ContainerStarted","Data":"2bab75f7ab941c7424853625896c532d3d71d270d18a37622f767e28d6e64d86"}
Aug 04 18:02:05.688590 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.688225    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rwlh5"]
Aug 04 18:02:05.691421 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.691400    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.696224 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.695875    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 18:02:05.696224 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.695942    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 18:02:05.696224 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.695967    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 18:02:05.696224 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.696082    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zlmsf\""
Aug 04 18:02:05.697536 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.697070    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 18:02:05.697536 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.697267    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 18:02:05.697536 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.697460    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 18:02:05.855069 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.854938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20f5604-4e33-498e-8c6f-ae34999deff6-metrics-client-ca\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855069 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.854998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phbl8\" (UniqueName: \"kubernetes.io/projected/e20f5604-4e33-498e-8c6f-ae34999deff6-kube-api-access-phbl8\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855069 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.855028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-root\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855382 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.855109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-wtmp\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855382 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.855165    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855382 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.855237    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-tls\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855382 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.855264    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855382 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.855298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-textfile\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.855382 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.855325    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-sys\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-tls\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-textfile\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956368 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-sys\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956368 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20f5604-4e33-498e-8c6f-ae34999deff6-metrics-client-ca\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956368 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phbl8\" (UniqueName: \"kubernetes.io/projected/e20f5604-4e33-498e-8c6f-ae34999deff6-kube-api-access-phbl8\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956368 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956242    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-root\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956368 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-wtmp\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956368 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956719 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-textfile\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956719 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-root\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956719 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-sys\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956872 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-wtmp\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.956958 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.956931    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.957174 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.957122    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20f5604-4e33-498e-8c6f-ae34999deff6-metrics-client-ca\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.958838 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.958808    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-tls\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.958940 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.958913    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e20f5604-4e33-498e-8c6f-ae34999deff6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:05.967320 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:05.967296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phbl8\" (UniqueName: \"kubernetes.io/projected/e20f5604-4e33-498e-8c6f-ae34999deff6-kube-api-access-phbl8\") pod \"node-exporter-rwlh5\" (UID: \"e20f5604-4e33-498e-8c6f-ae34999deff6\") " pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:06.005302 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:06.005267    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rwlh5"
Aug 04 18:02:09.553046 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:02:09.553022    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20f5604_4e33_498e_8c6f_ae34999deff6.slice/crio-df19edaed602afdc178c1241c8ae7c73a72ee860af4e0ff0c8149fe65fca35c2 WatchSource:0}: Error finding container df19edaed602afdc178c1241c8ae7c73a72ee860af4e0ff0c8149fe65fca35c2: Status 404 returned error can't find the container with id df19edaed602afdc178c1241c8ae7c73a72ee860af4e0ff0c8149fe65fca35c2
Aug 04 18:02:10.212546 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:10.212488    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-97psv" event={"ID":"3c108d6c-e58c-4ce8-9d24-9033bb48d08c","Type":"ContainerStarted","Data":"1a6bf670591d061f37ef26a39a1a6bcdb87b605d96b0886fc803be105eab933b"}
Aug 04 18:02:10.212796 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:10.212710    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-97psv"
Aug 04 18:02:10.214076 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:10.214013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwlh5" event={"ID":"e20f5604-4e33-498e-8c6f-ae34999deff6","Type":"ContainerStarted","Data":"df19edaed602afdc178c1241c8ae7c73a72ee860af4e0ff0c8149fe65fca35c2"}
Aug 04 18:02:10.216397 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:10.216372    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jlxm4" event={"ID":"664dc729-b7a8-4f4d-9fd6-fbc54f752945","Type":"ContainerStarted","Data":"a6bb4e0f0f73bf6892b8461db8f22ffc25e31aa437bb1b4f1d3a347c1a0f3627"}
Aug 04 18:02:10.228911 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:10.228886    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-97psv"
Aug 04 18:02:10.237383 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:10.237333    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-97psv" podStartSLOduration=1.8725960160000001 podStartE2EDuration="19.23731672s" totalImagesPullingTime="17.364720704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:02:10.235535989 +0000 UTC m=+89.917984320" watchObservedRunningTime="2026-08-04 18:02:10.23731672 +0000 UTC m=+89.919765039"
Aug 04 18:02:10.261595 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:10.261541    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jlxm4" podStartSLOduration=36.478740414 podStartE2EDuration="48.261526594s" totalImagesPullingTime="11.78278618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:01:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:02:10.260197877 +0000 UTC m=+89.942646190" watchObservedRunningTime="2026-08-04 18:02:10.261526594 +0000 UTC m=+89.943974911"
Aug 04 18:02:12.117384 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:12.117332    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:02:13.236868 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:13.236833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwlh5" event={"ID":"e20f5604-4e33-498e-8c6f-ae34999deff6","Type":"ContainerStarted","Data":"7fd0e3b308d149c393830fd6d541a181d1f9026292926b68a6c308306c18bdf2"}
Aug 04 18:02:14.132483 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:14.132456    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55c9667b59-4dzrf"
Aug 04 18:02:14.243128 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:14.243086    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20f5604-4e33-498e-8c6f-ae34999deff6" containerID="7fd0e3b308d149c393830fd6d541a181d1f9026292926b68a6c308306c18bdf2" exitCode=0
Aug 04 18:02:14.243583 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:14.243173    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwlh5" event={"ID":"e20f5604-4e33-498e-8c6f-ae34999deff6","Type":"ContainerDied","Data":"7fd0e3b308d149c393830fd6d541a181d1f9026292926b68a6c308306c18bdf2"}
Aug 04 18:02:15.249849 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:15.249811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwlh5" event={"ID":"e20f5604-4e33-498e-8c6f-ae34999deff6","Type":"ContainerStarted","Data":"f99f84fe8415cf28b2ea0f1db3bde68e38359ef560be2b5b4f9bdb866481e043"}
Aug 04 18:02:15.250363 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:15.249870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwlh5" event={"ID":"e20f5604-4e33-498e-8c6f-ae34999deff6","Type":"ContainerStarted","Data":"51f397ab7e5c352a9410c37f1690379380256b1a2225c86d9e82958f9fb9d238"}
Aug 04 18:02:17.131696 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.131639    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86bd98686-jnm59" podUID="7ea64979-84ea-4b18-abb4-3b5e5adb5670" containerName="registry" containerID="cri-o://e45021350d0a190d6926c793f4c454bc3a2c80c7a09f310465bd9ecd4dc2bdad" gracePeriod=30
Aug 04 18:02:17.261057 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.261025    2562 generic.go:356] "Generic (PLEG): container finished" podID="7ea64979-84ea-4b18-abb4-3b5e5adb5670" containerID="e45021350d0a190d6926c793f4c454bc3a2c80c7a09f310465bd9ecd4dc2bdad" exitCode=0
Aug 04 18:02:17.261174 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.261092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bd98686-jnm59" event={"ID":"7ea64979-84ea-4b18-abb4-3b5e5adb5670","Type":"ContainerDied","Data":"e45021350d0a190d6926c793f4c454bc3a2c80c7a09f310465bd9ecd4dc2bdad"}
Aug 04 18:02:17.429953 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.429926    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:02:17.457002 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.456939    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-image-registry-private-configuration\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.457002 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.457002    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.457229 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.457032    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7ea64979-84ea-4b18-abb4-3b5e5adb5670-ca-trust-extracted\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.457229 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.457059    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-installation-pull-secrets\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.457229 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.457091    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-certificates\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.457229 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.457137    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-trusted-ca\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.457229 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.457180    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-bound-sa-token\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.457229 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.457216    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvlgq\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-kube-api-access-jvlgq\") pod \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\" (UID: \"7ea64979-84ea-4b18-abb4-3b5e5adb5670\") "
Aug 04 18:02:17.458696 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.458402    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:02:17.461837 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.459789    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rwlh5" podStartSLOduration=8.86169341 podStartE2EDuration="12.459769103s" totalImagesPullingTime="3.501119358s" totalInitContainerRuntime="96.956335ms" isStatefulPod=true podCreationTimestamp="2026-08-04 18:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:02:15.279730144 +0000 UTC m=+94.962178461" watchObservedRunningTime="2026-08-04 18:02:17.459769103 +0000 UTC m=+97.142217423"
Aug 04 18:02:17.461837 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.460711    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:02:17.461837 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.462316    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:02:17.461837 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.463181    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-kube-api-access-jvlgq" (OuterVolumeSpecName: "kube-api-access-jvlgq") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "kube-api-access-jvlgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:02:17.461837 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.464233    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:02:17.468359 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.468331    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:02:17.469321 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.469291    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:02:17.469697 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.469668    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea64979-84ea-4b18-abb4-3b5e5adb5670-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7ea64979-84ea-4b18-abb4-3b5e5adb5670" (UID: "7ea64979-84ea-4b18-abb4-3b5e5adb5670"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:02:17.558086 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558050    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:17.558086 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558079    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7ea64979-84ea-4b18-abb4-3b5e5adb5670-ca-trust-extracted\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:17.558289 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558095    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-installation-pull-secrets\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:17.558289 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558109    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-registry-certificates\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:17.558289 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558119    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ea64979-84ea-4b18-abb4-3b5e5adb5670-trusted-ca\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:17.558289 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558129    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-bound-sa-token\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:17.558289 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558136    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvlgq\" (UniqueName: \"kubernetes.io/projected/7ea64979-84ea-4b18-abb4-3b5e5adb5670-kube-api-access-jvlgq\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:17.558289 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:17.558149    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7ea64979-84ea-4b18-abb4-3b5e5adb5670-image-registry-private-configuration\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:02:18.087895 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:18.087862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rvmpt"
Aug 04 18:02:18.267454 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:18.267411    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bd98686-jnm59" event={"ID":"7ea64979-84ea-4b18-abb4-3b5e5adb5670","Type":"ContainerDied","Data":"f0b45a10898d57203b647b664dfcfe11934ae3d25047bba87785a8cfe6579e90"}
Aug 04 18:02:18.267454 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:18.267441    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bd98686-jnm59"
Aug 04 18:02:18.267993 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:18.267536    2562 scope.go:117] "RemoveContainer" containerID="e45021350d0a190d6926c793f4c454bc3a2c80c7a09f310465bd9ecd4dc2bdad"
Aug 04 18:02:18.292281 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:18.292248    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86bd98686-jnm59"]
Aug 04 18:02:18.297235 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:18.297215    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86bd98686-jnm59"]
Aug 04 18:02:18.808433 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:18.808398    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ea64979-84ea-4b18-abb4-3b5e5adb5670" path="/var/lib/kubelet/pods/7ea64979-84ea-4b18-abb4-3b5e5adb5670/volumes"
Aug 04 18:02:34.329287 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:34.329253    2562 generic.go:356] "Generic (PLEG): container finished" podID="386c9d7a-67a4-432d-b226-97305d53924a" containerID="edfbf0465672bdb5ec66f23690b268a7c79870ed86203f2917db68f3a02dd3a4" exitCode=0
Aug 04 18:02:34.329701 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:34.329325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d" event={"ID":"386c9d7a-67a4-432d-b226-97305d53924a","Type":"ContainerDied","Data":"edfbf0465672bdb5ec66f23690b268a7c79870ed86203f2917db68f3a02dd3a4"}
Aug 04 18:02:34.329701 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:34.329634    2562 scope.go:117] "RemoveContainer" containerID="edfbf0465672bdb5ec66f23690b268a7c79870ed86203f2917db68f3a02dd3a4"
Aug 04 18:02:35.334240 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:35.334203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-xjm4d" event={"ID":"386c9d7a-67a4-432d-b226-97305d53924a","Type":"ContainerStarted","Data":"32afa1833298046445691c166a66f101eb69e6a0deed7dcf7345dad4d0ac1cdc"}
Aug 04 18:02:36.339897 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:36.339860    2562 generic.go:356] "Generic (PLEG): container finished" podID="fd44f2d2-2f07-492e-b98f-a0cc23ab3873" containerID="ee1a99a39557cd44a00fe412814bd47c48e98a3adb3a5c361d7ea4d25905541f" exitCode=0
Aug 04 18:02:36.339897 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:36.339884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn" event={"ID":"fd44f2d2-2f07-492e-b98f-a0cc23ab3873","Type":"ContainerDied","Data":"ee1a99a39557cd44a00fe412814bd47c48e98a3adb3a5c361d7ea4d25905541f"}
Aug 04 18:02:36.340252 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:36.340198    2562 scope.go:117] "RemoveContainer" containerID="ee1a99a39557cd44a00fe412814bd47c48e98a3adb3a5c361d7ea4d25905541f"
Aug 04 18:02:37.345049 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:02:37.345015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-4q8wn" event={"ID":"fd44f2d2-2f07-492e-b98f-a0cc23ab3873","Type":"ContainerStarted","Data":"317fa8c014cc99988def794abf7b7f4be44485167570416492b5dec9c755c59f"}
Aug 04 18:05:40.669816 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:05:40.669783    2562 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 18:06:54.055193 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.055148    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"]
Aug 04 18:06:54.055699 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.055453    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea64979-84ea-4b18-abb4-3b5e5adb5670" containerName="registry"
Aug 04 18:06:54.055699 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.055465    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea64979-84ea-4b18-abb4-3b5e5adb5670" containerName="registry"
Aug 04 18:06:54.055699 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.055529    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea64979-84ea-4b18-abb4-3b5e5adb5670" containerName="registry"
Aug 04 18:06:54.057460 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.057442    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.060202 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.060180    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Aug 04 18:06:54.060328 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.060241    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-rslqj\""
Aug 04 18:06:54.060328 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.060288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Aug 04 18:06:54.060415 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.060389    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Aug 04 18:06:54.069745 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.069723    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"]
Aug 04 18:06:54.193827 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.193799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2aedd402-745d-49f6-8d01-b75a050ddf41-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-m72ld\" (UID: \"2aedd402-745d-49f6-8d01-b75a050ddf41\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.193944 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.193850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7mg4\" (UniqueName: \"kubernetes.io/projected/2aedd402-745d-49f6-8d01-b75a050ddf41-kube-api-access-r7mg4\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-m72ld\" (UID: \"2aedd402-745d-49f6-8d01-b75a050ddf41\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.294891 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.294859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7mg4\" (UniqueName: \"kubernetes.io/projected/2aedd402-745d-49f6-8d01-b75a050ddf41-kube-api-access-r7mg4\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-m72ld\" (UID: \"2aedd402-745d-49f6-8d01-b75a050ddf41\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.295014 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.294918    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2aedd402-745d-49f6-8d01-b75a050ddf41-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-m72ld\" (UID: \"2aedd402-745d-49f6-8d01-b75a050ddf41\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.297350 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.297329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2aedd402-745d-49f6-8d01-b75a050ddf41-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-m72ld\" (UID: \"2aedd402-745d-49f6-8d01-b75a050ddf41\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.306753 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.306701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7mg4\" (UniqueName: \"kubernetes.io/projected/2aedd402-745d-49f6-8d01-b75a050ddf41-kube-api-access-r7mg4\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-m72ld\" (UID: \"2aedd402-745d-49f6-8d01-b75a050ddf41\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.367913 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.367886    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:54.497768 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.497735    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"]
Aug 04 18:06:54.500817 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:06:54.500786    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aedd402_745d_49f6_8d01_b75a050ddf41.slice/crio-e88cf2c17c187b1ce7f3d9b6237339c11666f9e1deef70b32edb3b57bb77ee4c WatchSource:0}: Error finding container e88cf2c17c187b1ce7f3d9b6237339c11666f9e1deef70b32edb3b57bb77ee4c: Status 404 returned error can't find the container with id e88cf2c17c187b1ce7f3d9b6237339c11666f9e1deef70b32edb3b57bb77ee4c
Aug 04 18:06:54.502597 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:54.502578    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:06:55.254924 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:55.254893    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld" event={"ID":"2aedd402-745d-49f6-8d01-b75a050ddf41","Type":"ContainerStarted","Data":"e88cf2c17c187b1ce7f3d9b6237339c11666f9e1deef70b32edb3b57bb77ee4c"}
Aug 04 18:06:58.270232 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:58.270198    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld" event={"ID":"2aedd402-745d-49f6-8d01-b75a050ddf41","Type":"ContainerStarted","Data":"7dafde46f1605e93ddc419f8e40bff49df8c7cae6beacc1bb3219933777fa474"}
Aug 04 18:06:58.270599 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:58.270328    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:06:58.292625 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:58.292575    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld" podStartSLOduration=1.027396186 podStartE2EDuration="4.292559083s" totalImagesPullingTime="3.265162897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:06:58.290423537 +0000 UTC m=+377.972871856" watchObservedRunningTime="2026-08-04 18:06:58.292559083 +0000 UTC m=+377.975007403"
Aug 04 18:06:59.048498 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.048455    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-87s6c"]
Aug 04 18:06:59.051384 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.051362    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.054314 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.054290    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Aug 04 18:06:59.054425 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.054292    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Aug 04 18:06:59.054641 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.054618    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-glqs2\""
Aug 04 18:06:59.064648 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.064627    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-87s6c"]
Aug 04 18:06:59.136825 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.136797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-certificates\") pod \"keda-admission-56f9798bb4-87s6c\" (UID: \"bcec665c-7a46-4252-9c9e-868973675abe\") " pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.136960 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.136843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62s8p\" (UniqueName: \"kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-kube-api-access-62s8p\") pod \"keda-admission-56f9798bb4-87s6c\" (UID: \"bcec665c-7a46-4252-9c9e-868973675abe\") " pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.238225 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.238198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-certificates\") pod \"keda-admission-56f9798bb4-87s6c\" (UID: \"bcec665c-7a46-4252-9c9e-868973675abe\") " pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.238421 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.238232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62s8p\" (UniqueName: \"kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-kube-api-access-62s8p\") pod \"keda-admission-56f9798bb4-87s6c\" (UID: \"bcec665c-7a46-4252-9c9e-868973675abe\") " pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.238421 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:06:59.238374    2562 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Aug 04 18:06:59.238421 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:06:59.238404    2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-87s6c: secret "keda-admission-webhooks-certs" not found
Aug 04 18:06:59.238553 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:06:59.238494    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-certificates podName:bcec665c-7a46-4252-9c9e-868973675abe nodeName:}" failed. No retries permitted until 2026-08-04 18:06:59.738473448 +0000 UTC m=+379.420921751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-certificates") pod "keda-admission-56f9798bb4-87s6c" (UID: "bcec665c-7a46-4252-9c9e-868973675abe") : secret "keda-admission-webhooks-certs" not found
Aug 04 18:06:59.249541 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.249517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62s8p\" (UniqueName: \"kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-kube-api-access-62s8p\") pod \"keda-admission-56f9798bb4-87s6c\" (UID: \"bcec665c-7a46-4252-9c9e-868973675abe\") " pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.741533 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.741488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-certificates\") pod \"keda-admission-56f9798bb4-87s6c\" (UID: \"bcec665c-7a46-4252-9c9e-868973675abe\") " pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.744299 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.744276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bcec665c-7a46-4252-9c9e-868973675abe-certificates\") pod \"keda-admission-56f9798bb4-87s6c\" (UID: \"bcec665c-7a46-4252-9c9e-868973675abe\") " pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:06:59.964262 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:06:59.964225    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:07:00.093378 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:07:00.093348    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-87s6c"]
Aug 04 18:07:00.096452 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:07:00.096422    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcec665c_7a46_4252_9c9e_868973675abe.slice/crio-b0a20f89141db9a2843e99a3554b7f6cb1f2d2ccc0dfb6c39b8facb60967d23c WatchSource:0}: Error finding container b0a20f89141db9a2843e99a3554b7f6cb1f2d2ccc0dfb6c39b8facb60967d23c: Status 404 returned error can't find the container with id b0a20f89141db9a2843e99a3554b7f6cb1f2d2ccc0dfb6c39b8facb60967d23c
Aug 04 18:07:00.279598 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:07:00.279521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-87s6c" event={"ID":"bcec665c-7a46-4252-9c9e-868973675abe","Type":"ContainerStarted","Data":"b0a20f89141db9a2843e99a3554b7f6cb1f2d2ccc0dfb6c39b8facb60967d23c"}
Aug 04 18:07:03.291849 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:07:03.291813    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-87s6c" event={"ID":"bcec665c-7a46-4252-9c9e-868973675abe","Type":"ContainerStarted","Data":"f63c8168f8b29a0efebbc1a147ef5595743c464e4094d2eed731edac9d980217"}
Aug 04 18:07:03.292269 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:07:03.291935    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:07:03.311459 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:07:03.311412    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-87s6c" podStartSLOduration=1.652157849 podStartE2EDuration="4.311399145s" totalImagesPullingTime="2.659241296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:06:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:07:03.309478851 +0000 UTC m=+382.991927206" watchObservedRunningTime="2026-08-04 18:07:03.311399145 +0000 UTC m=+382.993847463"
Aug 04 18:07:19.277185 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:07:19.277152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-m72ld"
Aug 04 18:07:24.298611 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:07:24.298581    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-87s6c"
Aug 04 18:08:19.248193 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.248118    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-mt52k"]
Aug 04 18:08:19.250201 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.250184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:19.252667 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.252643    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Aug 04 18:08:19.253043 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.253017    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-cg769\""
Aug 04 18:08:19.253949 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.253927    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Aug 04 18:08:19.254068 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.254032    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Aug 04 18:08:19.262899 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.262876    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-mt52k"]
Aug 04 18:08:19.281574 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.281547    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-drnvg"]
Aug 04 18:08:19.283926 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.283904    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.286708 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.286689    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Aug 04 18:08:19.286817 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.286695    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-wl8tr\""
Aug 04 18:08:19.292562 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.292542    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-drnvg"]
Aug 04 18:08:19.403123 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.403091    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpcgf\" (UniqueName: \"kubernetes.io/projected/809b24a5-9bd3-40a9-9320-b14f4bfa40a6-kube-api-access-hpcgf\") pod \"seaweedfs-64568bcd49-drnvg\" (UID: \"809b24a5-9bd3-40a9-9320-b14f4bfa40a6\") " pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.403123 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.403123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/809b24a5-9bd3-40a9-9320-b14f4bfa40a6-data\") pod \"seaweedfs-64568bcd49-drnvg\" (UID: \"809b24a5-9bd3-40a9-9320-b14f4bfa40a6\") " pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.403337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.403147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw98z\" (UniqueName: \"kubernetes.io/projected/f5294598-bb18-4c8f-ae88-bd4313a7311d-kube-api-access-tw98z\") pod \"llmisvc-controller-manager-654c557856-mt52k\" (UID: \"f5294598-bb18-4c8f-ae88-bd4313a7311d\") " pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:19.403337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.403168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5294598-bb18-4c8f-ae88-bd4313a7311d-cert\") pod \"llmisvc-controller-manager-654c557856-mt52k\" (UID: \"f5294598-bb18-4c8f-ae88-bd4313a7311d\") " pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:19.504476 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.504407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpcgf\" (UniqueName: \"kubernetes.io/projected/809b24a5-9bd3-40a9-9320-b14f4bfa40a6-kube-api-access-hpcgf\") pod \"seaweedfs-64568bcd49-drnvg\" (UID: \"809b24a5-9bd3-40a9-9320-b14f4bfa40a6\") " pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.504476 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.504435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/809b24a5-9bd3-40a9-9320-b14f4bfa40a6-data\") pod \"seaweedfs-64568bcd49-drnvg\" (UID: \"809b24a5-9bd3-40a9-9320-b14f4bfa40a6\") " pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.504476 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.504460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw98z\" (UniqueName: \"kubernetes.io/projected/f5294598-bb18-4c8f-ae88-bd4313a7311d-kube-api-access-tw98z\") pod \"llmisvc-controller-manager-654c557856-mt52k\" (UID: \"f5294598-bb18-4c8f-ae88-bd4313a7311d\") " pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:19.504704 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.504555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5294598-bb18-4c8f-ae88-bd4313a7311d-cert\") pod \"llmisvc-controller-manager-654c557856-mt52k\" (UID: \"f5294598-bb18-4c8f-ae88-bd4313a7311d\") " pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:19.504704 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:08:19.504663    2562 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Aug 04 18:08:19.504817 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:08:19.504727    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5294598-bb18-4c8f-ae88-bd4313a7311d-cert podName:f5294598-bb18-4c8f-ae88-bd4313a7311d nodeName:}" failed. No retries permitted until 2026-08-04 18:08:20.004709646 +0000 UTC m=+459.687157942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f5294598-bb18-4c8f-ae88-bd4313a7311d-cert") pod "llmisvc-controller-manager-654c557856-mt52k" (UID: "f5294598-bb18-4c8f-ae88-bd4313a7311d") : secret "llmisvc-webhook-server-cert" not found
Aug 04 18:08:19.504817 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.504762    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/809b24a5-9bd3-40a9-9320-b14f4bfa40a6-data\") pod \"seaweedfs-64568bcd49-drnvg\" (UID: \"809b24a5-9bd3-40a9-9320-b14f4bfa40a6\") " pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.514962 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.514939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpcgf\" (UniqueName: \"kubernetes.io/projected/809b24a5-9bd3-40a9-9320-b14f4bfa40a6-kube-api-access-hpcgf\") pod \"seaweedfs-64568bcd49-drnvg\" (UID: \"809b24a5-9bd3-40a9-9320-b14f4bfa40a6\") " pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.515064 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.515049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw98z\" (UniqueName: \"kubernetes.io/projected/f5294598-bb18-4c8f-ae88-bd4313a7311d-kube-api-access-tw98z\") pod \"llmisvc-controller-manager-654c557856-mt52k\" (UID: \"f5294598-bb18-4c8f-ae88-bd4313a7311d\") " pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:19.593113 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.593091    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:19.719577 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:19.719552    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-drnvg"]
Aug 04 18:08:20.008748 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:20.008716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5294598-bb18-4c8f-ae88-bd4313a7311d-cert\") pod \"llmisvc-controller-manager-654c557856-mt52k\" (UID: \"f5294598-bb18-4c8f-ae88-bd4313a7311d\") " pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:20.011119 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:20.011096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5294598-bb18-4c8f-ae88-bd4313a7311d-cert\") pod \"llmisvc-controller-manager-654c557856-mt52k\" (UID: \"f5294598-bb18-4c8f-ae88-bd4313a7311d\") " pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:20.160714 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:20.160681    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:20.314537 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:20.314511    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-mt52k"]
Aug 04 18:08:20.339128 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:08:20.339099    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf5294598_bb18_4c8f_ae88_bd4313a7311d.slice/crio-8ae53c2681a5309108edda6efc34bdaa968c8105143bb5760aa6ef4a28402967 WatchSource:0}: Error finding container 8ae53c2681a5309108edda6efc34bdaa968c8105143bb5760aa6ef4a28402967: Status 404 returned error can't find the container with id 8ae53c2681a5309108edda6efc34bdaa968c8105143bb5760aa6ef4a28402967
Aug 04 18:08:20.590016 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:20.589927    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-mt52k" event={"ID":"f5294598-bb18-4c8f-ae88-bd4313a7311d","Type":"ContainerStarted","Data":"8ae53c2681a5309108edda6efc34bdaa968c8105143bb5760aa6ef4a28402967"}
Aug 04 18:08:20.591266 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:20.591236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-drnvg" event={"ID":"809b24a5-9bd3-40a9-9320-b14f4bfa40a6","Type":"ContainerStarted","Data":"95253cbcd3b75ca12c09e3c38cbdb09d03fd5de560527caa8a06a847895b5e36"}
Aug 04 18:08:23.606758 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:23.606717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-mt52k" event={"ID":"f5294598-bb18-4c8f-ae88-bd4313a7311d","Type":"ContainerStarted","Data":"9c794224fffb2f20dc7352d87f231463313614b248c8570b0d6ce6000ed17f9d"}
Aug 04 18:08:23.607212 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:23.606816    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:08:23.608172 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:23.608144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-drnvg" event={"ID":"809b24a5-9bd3-40a9-9320-b14f4bfa40a6","Type":"ContainerStarted","Data":"9883e7979fe6425590248b6beebece9f07955dc413ad5f497293c0131bfc16d8"}
Aug 04 18:08:23.608301 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:23.608228    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:23.626328 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:23.626286    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-654c557856-mt52k" podStartSLOduration=1.7942368229999999 podStartE2EDuration="4.626272746s" totalImagesPullingTime="2.832035923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:08:23.625178016 +0000 UTC m=+463.307626333" watchObservedRunningTime="2026-08-04 18:08:23.626272746 +0000 UTC m=+463.308721096"
Aug 04 18:08:23.643433 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:23.643392    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-drnvg" podStartSLOduration=1.19502096 podStartE2EDuration="4.64337899s" totalImagesPullingTime="3.44835803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:08:23.642183087 +0000 UTC m=+463.324631405" watchObservedRunningTime="2026-08-04 18:08:23.64337899 +0000 UTC m=+463.325827308"
Aug 04 18:08:29.615280 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:29.615251    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-drnvg"
Aug 04 18:08:54.615085 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:08:54.615054    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-654c557856-mt52k"
Aug 04 18:11:24.551110 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.551073    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"]
Aug 04 18:11:24.553382 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.553365    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.556281 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.556252    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4a809-predictor-serving-cert\""
Aug 04 18:11:24.556399 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.556345    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4a809-kube-rbac-proxy-sar-config\""
Aug 04 18:11:24.556399 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.556383    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Aug 04 18:11:24.556522 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.556435    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-64gfv\""
Aug 04 18:11:24.557657 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.557640    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Aug 04 18:11:24.564753 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.564730    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"]
Aug 04 18:11:24.643011 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.642982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.643162 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.643026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjkfm\" (UniqueName: \"kubernetes.io/projected/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-api-access-mjkfm\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.643162 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.643051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab525bdc-980c-4be2-bd2d-4795da07000c-proxy-tls\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.743437 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.743406    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjkfm\" (UniqueName: \"kubernetes.io/projected/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-api-access-mjkfm\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.743626 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.743444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab525bdc-980c-4be2-bd2d-4795da07000c-proxy-tls\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.743626 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.743510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.744431 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.744411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.746092 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.746070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab525bdc-980c-4be2-bd2d-4795da07000c-proxy-tls\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.755332 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.755307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjkfm\" (UniqueName: \"kubernetes.io/projected/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-api-access-mjkfm\") pod \"error-404-isvc-4a809-predictor-6db7b75455-g9d8f\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") " pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.864294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.864234    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:24.995760 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:24.995730    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"]
Aug 04 18:11:24.998857 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:11:24.998830    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab525bdc_980c_4be2_bd2d_4795da07000c.slice/crio-2a60678765e484efe8192abb4833e3f5fdaea58f6e9eb805b611941aa25b712d WatchSource:0}: Error finding container 2a60678765e484efe8192abb4833e3f5fdaea58f6e9eb805b611941aa25b712d: Status 404 returned error can't find the container with id 2a60678765e484efe8192abb4833e3f5fdaea58f6e9eb805b611941aa25b712d
Aug 04 18:11:25.330804 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:25.330774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" event={"ID":"ab525bdc-980c-4be2-bd2d-4795da07000c","Type":"ContainerStarted","Data":"2a60678765e484efe8192abb4833e3f5fdaea58f6e9eb805b611941aa25b712d"}
Aug 04 18:11:36.388987 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:36.388950    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" event={"ID":"ab525bdc-980c-4be2-bd2d-4795da07000c","Type":"ContainerStarted","Data":"fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be"}
Aug 04 18:11:38.400089 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:38.400051    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" event={"ID":"ab525bdc-980c-4be2-bd2d-4795da07000c","Type":"ContainerStarted","Data":"70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943"}
Aug 04 18:11:38.400527 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:38.400120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:38.400527 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:38.400158    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:38.401784 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:38.401747    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Aug 04 18:11:38.420656 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:38.420601    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podStartSLOduration=1.166754258 podStartE2EDuration="14.420584871s" totalImagesPullingTime="13.253830613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:11:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:11:38.41978312 +0000 UTC m=+658.102231439" watchObservedRunningTime="2026-08-04 18:11:38.420584871 +0000 UTC m=+658.103033189"
Aug 04 18:11:39.404925 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:39.404874    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Aug 04 18:11:44.410655 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:44.410622    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:11:44.411333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:44.411301    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Aug 04 18:11:54.412248 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:11:54.412206    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Aug 04 18:12:04.411634 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:04.411585    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Aug 04 18:12:14.411380 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:14.411334    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Aug 04 18:12:24.411656 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:24.411622    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:12:54.558805 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.558772    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"]
Aug 04 18:12:54.559262 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.559011    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container" containerID="cri-o://fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be" gracePeriod=30
Aug 04 18:12:54.559262 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.559049    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kube-rbac-proxy" containerID="cri-o://70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943" gracePeriod=30
Aug 04 18:12:54.729178 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.729143    2562 generic.go:356] "Generic (PLEG): container finished" podID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerID="70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943" exitCode=2
Aug 04 18:12:54.729178 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.729188    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" event={"ID":"ab525bdc-980c-4be2-bd2d-4795da07000c","Type":"ContainerDied","Data":"70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943"}
Aug 04 18:12:54.781883 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.781850    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"]
Aug 04 18:12:54.784271 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.784249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.786912 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.786892    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-01f07-predictor-serving-cert\""
Aug 04 18:12:54.786912 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.786903    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-01f07-kube-rbac-proxy-sar-config\""
Aug 04 18:12:54.798168 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.798146    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"]
Aug 04 18:12:54.879088 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.879018    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.879088 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.879075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad732c12-b82a-4e00-82ee-3002aac085e5-proxy-tls\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.879269 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.879153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4jjb\" (UniqueName: \"kubernetes.io/projected/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-api-access-s4jjb\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.979676 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.979646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad732c12-b82a-4e00-82ee-3002aac085e5-proxy-tls\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.979803 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.979688    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4jjb\" (UniqueName: \"kubernetes.io/projected/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-api-access-s4jjb\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.979803 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.979724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.980412 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.980392    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.982114 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.982095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad732c12-b82a-4e00-82ee-3002aac085e5-proxy-tls\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:54.989040 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:54.989016    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4jjb\" (UniqueName: \"kubernetes.io/projected/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-api-access-s4jjb\") pod \"error-404-isvc-01f07-predictor-5758564b9f-55hk9\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") " pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:55.093947 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:55.093926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:55.221413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:55.221390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"]
Aug 04 18:12:55.223276 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:12:55.223252    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad732c12_b82a_4e00_82ee_3002aac085e5.slice/crio-acb03bc99bc892d8efaf42e6e5ce9ba3c56b78c901991da58fa4ca232b4f6b47 WatchSource:0}: Error finding container acb03bc99bc892d8efaf42e6e5ce9ba3c56b78c901991da58fa4ca232b4f6b47: Status 404 returned error can't find the container with id acb03bc99bc892d8efaf42e6e5ce9ba3c56b78c901991da58fa4ca232b4f6b47
Aug 04 18:12:55.735851 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:55.735812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" event={"ID":"ad732c12-b82a-4e00-82ee-3002aac085e5","Type":"ContainerStarted","Data":"c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6"}
Aug 04 18:12:55.736276 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:55.735877    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" event={"ID":"ad732c12-b82a-4e00-82ee-3002aac085e5","Type":"ContainerStarted","Data":"1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944"}
Aug 04 18:12:55.736276 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:55.735906    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" event={"ID":"ad732c12-b82a-4e00-82ee-3002aac085e5","Type":"ContainerStarted","Data":"acb03bc99bc892d8efaf42e6e5ce9ba3c56b78c901991da58fa4ca232b4f6b47"}
Aug 04 18:12:55.736276 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:55.735941    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:55.760904 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:55.760854    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podStartSLOduration=1.7608385279999998 podStartE2EDuration="1.760838528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:12:55.759295212 +0000 UTC m=+735.441743553" watchObservedRunningTime="2026-08-04 18:12:55.760838528 +0000 UTC m=+735.443286847"
Aug 04 18:12:56.740821 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:56.740794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:12:56.741841 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:56.741805    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 04 18:12:57.510646 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.510621    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:12:57.599433 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.599347    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab525bdc-980c-4be2-bd2d-4795da07000c-proxy-tls\") pod \"ab525bdc-980c-4be2-bd2d-4795da07000c\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") "
Aug 04 18:12:57.599433 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.599420    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-rbac-proxy-sar-config\") pod \"ab525bdc-980c-4be2-bd2d-4795da07000c\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") "
Aug 04 18:12:57.599663 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.599452    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjkfm\" (UniqueName: \"kubernetes.io/projected/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-api-access-mjkfm\") pod \"ab525bdc-980c-4be2-bd2d-4795da07000c\" (UID: \"ab525bdc-980c-4be2-bd2d-4795da07000c\") "
Aug 04 18:12:57.599801 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.599773    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ab525bdc-980c-4be2-bd2d-4795da07000c" (UID: "ab525bdc-980c-4be2-bd2d-4795da07000c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:12:57.601625 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.601599    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab525bdc-980c-4be2-bd2d-4795da07000c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ab525bdc-980c-4be2-bd2d-4795da07000c" (UID: "ab525bdc-980c-4be2-bd2d-4795da07000c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:12:57.601727 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.601671    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-api-access-mjkfm" (OuterVolumeSpecName: "kube-api-access-mjkfm") pod "ab525bdc-980c-4be2-bd2d-4795da07000c" (UID: "ab525bdc-980c-4be2-bd2d-4795da07000c"). InnerVolumeSpecName "kube-api-access-mjkfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:12:57.699975 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.699938    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab525bdc-980c-4be2-bd2d-4795da07000c-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:12:57.699975 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.699974    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:12:57.699975 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.699988    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjkfm\" (UniqueName: \"kubernetes.io/projected/ab525bdc-980c-4be2-bd2d-4795da07000c-kube-api-access-mjkfm\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:12:57.746365 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.746324    2562 generic.go:356] "Generic (PLEG): container finished" podID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerID="fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be" exitCode=0
Aug 04 18:12:57.746789 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.746373    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"
Aug 04 18:12:57.746789 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.746383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" event={"ID":"ab525bdc-980c-4be2-bd2d-4795da07000c","Type":"ContainerDied","Data":"fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be"}
Aug 04 18:12:57.746789 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.746438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f" event={"ID":"ab525bdc-980c-4be2-bd2d-4795da07000c","Type":"ContainerDied","Data":"2a60678765e484efe8192abb4833e3f5fdaea58f6e9eb805b611941aa25b712d"}
Aug 04 18:12:57.746789 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.746465    2562 scope.go:117] "RemoveContainer" containerID="70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943"
Aug 04 18:12:57.746992 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.746913    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 04 18:12:57.757223 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.757203    2562 scope.go:117] "RemoveContainer" containerID="fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be"
Aug 04 18:12:57.764866 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.764849    2562 scope.go:117] "RemoveContainer" containerID="70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943"
Aug 04 18:12:57.765111 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:12:57.765090    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943\": container with ID starting with 70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943 not found: ID does not exist" containerID="70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943"
Aug 04 18:12:57.765156 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.765118    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943"} err="failed to get container status \"70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943\": rpc error: code = NotFound desc = could not find container \"70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943\": container with ID starting with 70445d077aad8c352611e67d0295f180b9ec1b120526a8d94cb7036608212943 not found: ID does not exist"
Aug 04 18:12:57.765156 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.765146    2562 scope.go:117] "RemoveContainer" containerID="fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be"
Aug 04 18:12:57.765391 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:12:57.765372    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be\": container with ID starting with fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be not found: ID does not exist" containerID="fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be"
Aug 04 18:12:57.765450 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.765395    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be"} err="failed to get container status \"fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be\": rpc error: code = NotFound desc = could not find container \"fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be\": container with ID starting with fca0792bb2c73c110b8165de184d92dd66c0cf0257109ab66659bd260357c2be not found: ID does not exist"
Aug 04 18:12:57.777901 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.777878    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"]
Aug 04 18:12:57.785311 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:57.785288    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4a809-predictor-6db7b75455-g9d8f"]
Aug 04 18:12:58.812466 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:12:58.812435    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" path="/var/lib/kubelet/pods/ab525bdc-980c-4be2-bd2d-4795da07000c/volumes"
Aug 04 18:13:02.751345 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:02.751317    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:13:02.751824 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:02.751796    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 04 18:13:12.751763 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:12.751724    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 04 18:13:22.752347 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:22.752308    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 04 18:13:32.751906 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:32.751866    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 04 18:13:42.752655 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:42.752617    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:13:54.666894 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.666863    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"]
Aug 04 18:13:54.667319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.667172    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kube-rbac-proxy"
Aug 04 18:13:54.667319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.667183    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kube-rbac-proxy"
Aug 04 18:13:54.667319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.667192    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container"
Aug 04 18:13:54.667319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.667198    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container"
Aug 04 18:13:54.667319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.667247    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kserve-container"
Aug 04 18:13:54.667319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.667254    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab525bdc-980c-4be2-bd2d-4795da07000c" containerName="kube-rbac-proxy"
Aug 04 18:13:54.671433 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.671415    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.674311 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.674289    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-315c2-kube-rbac-proxy-sar-config\""
Aug 04 18:13:54.674413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.674289    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-315c2-predictor-serving-cert\""
Aug 04 18:13:54.680259 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.680238    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"]
Aug 04 18:13:54.717926 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.717898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gtd6\" (UniqueName: \"kubernetes.io/projected/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-api-access-4gtd6\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.718062 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.717940    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7047780-2249-44f0-a3e4-e610c0b56f50-proxy-tls\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.718062 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.717992    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.818737 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.818701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7047780-2249-44f0-a3e4-e610c0b56f50-proxy-tls\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.818881 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.818758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.818881 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.818797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gtd6\" (UniqueName: \"kubernetes.io/projected/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-api-access-4gtd6\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.819445 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.819419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.821058 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.821041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7047780-2249-44f0-a3e4-e610c0b56f50-proxy-tls\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.827933 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.827912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gtd6\" (UniqueName: \"kubernetes.io/projected/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-api-access-4gtd6\") pod \"error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") " pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:54.982417 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:54.982364    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:55.113065 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:55.113041    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"]
Aug 04 18:13:55.115216 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:13:55.115187    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7047780_2249_44f0_a3e4_e610c0b56f50.slice/crio-dd5e8ee2df635dd0460e5a98d4634af9a491f6ccb151a1ff08fc411557419943 WatchSource:0}: Error finding container dd5e8ee2df635dd0460e5a98d4634af9a491f6ccb151a1ff08fc411557419943: Status 404 returned error can't find the container with id dd5e8ee2df635dd0460e5a98d4634af9a491f6ccb151a1ff08fc411557419943
Aug 04 18:13:55.979901 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:55.979871    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" event={"ID":"f7047780-2249-44f0-a3e4-e610c0b56f50","Type":"ContainerStarted","Data":"0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787"}
Aug 04 18:13:55.980261 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:55.979925    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" event={"ID":"f7047780-2249-44f0-a3e4-e610c0b56f50","Type":"ContainerStarted","Data":"10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2"}
Aug 04 18:13:55.980261 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:55.979948    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" event={"ID":"f7047780-2249-44f0-a3e4-e610c0b56f50","Type":"ContainerStarted","Data":"dd5e8ee2df635dd0460e5a98d4634af9a491f6ccb151a1ff08fc411557419943"}
Aug 04 18:13:55.980261 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:55.980029    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:56.003012 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:56.002966    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podStartSLOduration=2.00295309 podStartE2EDuration="2.00295309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:13:56.001914696 +0000 UTC m=+795.684363015" watchObservedRunningTime="2026-08-04 18:13:56.00295309 +0000 UTC m=+795.685401477"
Aug 04 18:13:56.983772 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:56.983734    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:13:56.984862 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:56.984836    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 04 18:13:57.993096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:13:57.993058    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 04 18:14:02.997560 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:14:02.997531    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:14:02.998121 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:14:02.998092    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 04 18:14:12.998166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:14:12.998129    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 04 18:14:22.998417 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:14:22.998373    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 04 18:14:32.998399 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:14:32.998359    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 04 18:14:42.998603 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:14:42.998569    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:22:09.760086 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.760052    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"]
Aug 04 18:22:09.763571 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.763546    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:09.765149 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.765125    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"]
Aug 04 18:22:09.765347 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.765324    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" containerID="cri-o://1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944" gracePeriod=30
Aug 04 18:22:09.765425 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.765405    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kube-rbac-proxy" containerID="cri-o://c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6" gracePeriod=30
Aug 04 18:22:09.766431 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.766407    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a1525-kube-rbac-proxy-sar-config\""
Aug 04 18:22:09.766546 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.766415    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a1525-predictor-serving-cert\""
Aug 04 18:22:09.776960 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.776929    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"]
Aug 04 18:22:09.947273 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.947233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlvkm\" (UniqueName: \"kubernetes.io/projected/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-api-access-mlvkm\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:09.947466 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.947294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-proxy-tls\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:09.947466 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:09.947391    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.048413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.048293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-proxy-tls\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.048413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.048371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.048668 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.048435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlvkm\" (UniqueName: \"kubernetes.io/projected/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-api-access-mlvkm\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.049179 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.049157    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.050842 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.050818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-proxy-tls\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.058802 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.058778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlvkm\" (UniqueName: \"kubernetes.io/projected/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-api-access-mlvkm\") pod \"error-404-isvc-a1525-predictor-7f846d5f79-7nw6m\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") " pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.079201 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.079166    2562 generic.go:356] "Generic (PLEG): container finished" podID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerID="c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6" exitCode=2
Aug 04 18:22:10.079354 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.079235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" event={"ID":"ad732c12-b82a-4e00-82ee-3002aac085e5","Type":"ContainerDied","Data":"c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6"}
Aug 04 18:22:10.085328 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.085306    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:10.229639 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:10.229598    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"]
Aug 04 18:22:10.234310 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:22:10.234272    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c872a0_542c_4a67_9fa8_d9a4e5c69e41.slice/crio-a7a11a417f152742835cdf7f7e6b5d9c631f690ba8dc5d6031ae3e12a21d0ee8 WatchSource:0}: Error finding container a7a11a417f152742835cdf7f7e6b5d9c631f690ba8dc5d6031ae3e12a21d0ee8: Status 404 returned error can't find the container with id a7a11a417f152742835cdf7f7e6b5d9c631f690ba8dc5d6031ae3e12a21d0ee8
Aug 04 18:22:11.085061 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:11.085021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" event={"ID":"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41","Type":"ContainerStarted","Data":"e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c"}
Aug 04 18:22:11.085428 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:11.085095    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" event={"ID":"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41","Type":"ContainerStarted","Data":"81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e"}
Aug 04 18:22:11.085428 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:11.085128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" event={"ID":"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41","Type":"ContainerStarted","Data":"a7a11a417f152742835cdf7f7e6b5d9c631f690ba8dc5d6031ae3e12a21d0ee8"}
Aug 04 18:22:11.085428 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:11.085251    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:11.105851 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:11.105798    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podStartSLOduration=2.10576755 podStartE2EDuration="2.10576755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:22:11.105219118 +0000 UTC m=+1290.787667437" watchObservedRunningTime="2026-08-04 18:22:11.10576755 +0000 UTC m=+1290.788215867"
Aug 04 18:22:12.089821 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:12.089787    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:12.091210 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:12.091176    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 04 18:22:12.747970 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:12.747913    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Aug 04 18:22:12.752646 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:12.752611    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 04 18:22:13.094596 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.094554    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 04 18:22:13.424446 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.424412    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:22:13.478117 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.478080    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-rbac-proxy-sar-config\") pod \"ad732c12-b82a-4e00-82ee-3002aac085e5\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") "
Aug 04 18:22:13.478330 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.478137    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4jjb\" (UniqueName: \"kubernetes.io/projected/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-api-access-s4jjb\") pod \"ad732c12-b82a-4e00-82ee-3002aac085e5\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") "
Aug 04 18:22:13.478450 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.478423    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ad732c12-b82a-4e00-82ee-3002aac085e5" (UID: "ad732c12-b82a-4e00-82ee-3002aac085e5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:22:13.480253 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.480222    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-api-access-s4jjb" (OuterVolumeSpecName: "kube-api-access-s4jjb") pod "ad732c12-b82a-4e00-82ee-3002aac085e5" (UID: "ad732c12-b82a-4e00-82ee-3002aac085e5"). InnerVolumeSpecName "kube-api-access-s4jjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:22:13.578830 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.578787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad732c12-b82a-4e00-82ee-3002aac085e5-proxy-tls\") pod \"ad732c12-b82a-4e00-82ee-3002aac085e5\" (UID: \"ad732c12-b82a-4e00-82ee-3002aac085e5\") "
Aug 04 18:22:13.579034 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.578951    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:22:13.579034 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.578962    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4jjb\" (UniqueName: \"kubernetes.io/projected/ad732c12-b82a-4e00-82ee-3002aac085e5-kube-api-access-s4jjb\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:22:13.580789 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.580754    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad732c12-b82a-4e00-82ee-3002aac085e5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ad732c12-b82a-4e00-82ee-3002aac085e5" (UID: "ad732c12-b82a-4e00-82ee-3002aac085e5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:22:13.679402 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:13.679291    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad732c12-b82a-4e00-82ee-3002aac085e5-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:22:14.100580 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.100546    2562 generic.go:356] "Generic (PLEG): container finished" podID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerID="1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944" exitCode=0
Aug 04 18:22:14.101005 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.100594    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"
Aug 04 18:22:14.101005 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.100636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" event={"ID":"ad732c12-b82a-4e00-82ee-3002aac085e5","Type":"ContainerDied","Data":"1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944"}
Aug 04 18:22:14.101005 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.100696    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9" event={"ID":"ad732c12-b82a-4e00-82ee-3002aac085e5","Type":"ContainerDied","Data":"acb03bc99bc892d8efaf42e6e5ce9ba3c56b78c901991da58fa4ca232b4f6b47"}
Aug 04 18:22:14.101005 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.100726    2562 scope.go:117] "RemoveContainer" containerID="c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6"
Aug 04 18:22:14.110296 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.110277    2562 scope.go:117] "RemoveContainer" containerID="1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944"
Aug 04 18:22:14.118264 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.118246    2562 scope.go:117] "RemoveContainer" containerID="c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6"
Aug 04 18:22:14.118553 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:22:14.118531    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6\": container with ID starting with c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6 not found: ID does not exist" containerID="c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6"
Aug 04 18:22:14.118638 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.118567    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6"} err="failed to get container status \"c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6\": rpc error: code = NotFound desc = could not find container \"c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6\": container with ID starting with c67cdb08ffd5ba512aece7c21b2cce2a04c05fdd092341f87a9a831b26d71fe6 not found: ID does not exist"
Aug 04 18:22:14.118638 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.118593    2562 scope.go:117] "RemoveContainer" containerID="1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944"
Aug 04 18:22:14.118850 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:22:14.118829    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944\": container with ID starting with 1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944 not found: ID does not exist" containerID="1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944"
Aug 04 18:22:14.118887 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.118856    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944"} err="failed to get container status \"1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944\": rpc error: code = NotFound desc = could not find container \"1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944\": container with ID starting with 1dc091b45e946f20ded11bd4140304316ab5c1ae26637ebeb3d1eba3f8b53944 not found: ID does not exist"
Aug 04 18:22:14.126263 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.126240    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"]
Aug 04 18:22:14.131251 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.131228    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01f07-predictor-5758564b9f-55hk9"]
Aug 04 18:22:14.808493 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:14.808454    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" path="/var/lib/kubelet/pods/ad732c12-b82a-4e00-82ee-3002aac085e5/volumes"
Aug 04 18:22:18.100360 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:18.100326    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:22:18.100998 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:18.100967    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 04 18:22:28.101820 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:28.101776    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 04 18:22:38.100908 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:38.100867    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 04 18:22:48.101796 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:48.101738    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 04 18:22:58.102328 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:22:58.102285    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:23:09.527912 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.527821    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"]
Aug 04 18:23:09.528337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.528170    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" containerID="cri-o://10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2" gracePeriod=30
Aug 04 18:23:09.528337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.528170    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kube-rbac-proxy" containerID="cri-o://0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787" gracePeriod=30
Aug 04 18:23:09.640791 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.640750    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"]
Aug 04 18:23:09.641228 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.641209    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container"
Aug 04 18:23:09.641294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.641231    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container"
Aug 04 18:23:09.641294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.641256    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kube-rbac-proxy"
Aug 04 18:23:09.641294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.641262    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kube-rbac-proxy"
Aug 04 18:23:09.641437 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.641310    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kube-rbac-proxy"
Aug 04 18:23:09.641437 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.641317    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad732c12-b82a-4e00-82ee-3002aac085e5" containerName="kserve-container"
Aug 04 18:23:09.645654 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.645628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.648552 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.648523    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-94d00-kube-rbac-proxy-sar-config\""
Aug 04 18:23:09.648695 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.648573    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-94d00-predictor-serving-cert\""
Aug 04 18:23:09.655439 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.655408    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"]
Aug 04 18:23:09.749918 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.749873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a240ae88-abdc-47c9-977c-f6b3fd943a01-proxy-tls\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.750138 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.749947    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnntp\" (UniqueName: \"kubernetes.io/projected/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-api-access-lnntp\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.750138 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.749981    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.851408 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.851296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a240ae88-abdc-47c9-977c-f6b3fd943a01-proxy-tls\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.851408 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.851375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnntp\" (UniqueName: \"kubernetes.io/projected/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-api-access-lnntp\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.851408 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.851407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.852202 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.852177    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.854005 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.853972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a240ae88-abdc-47c9-977c-f6b3fd943a01-proxy-tls\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.861543 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.861488    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnntp\" (UniqueName: \"kubernetes.io/projected/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-api-access-lnntp\") pod \"error-404-isvc-94d00-predictor-558f596f5c-5fxt9\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") " pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:09.960254 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:09.960209    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:10.128452 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.128334    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"]
Aug 04 18:23:10.131541 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:23:10.131493    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda240ae88_abdc_47c9_977c_f6b3fd943a01.slice/crio-006f5dcb2317c7e85e73ab16c48047a0e99120af217ea6663bfc5f8199aec824 WatchSource:0}: Error finding container 006f5dcb2317c7e85e73ab16c48047a0e99120af217ea6663bfc5f8199aec824: Status 404 returned error can't find the container with id 006f5dcb2317c7e85e73ab16c48047a0e99120af217ea6663bfc5f8199aec824
Aug 04 18:23:10.347221 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.347186    2562 generic.go:356] "Generic (PLEG): container finished" podID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerID="0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787" exitCode=2
Aug 04 18:23:10.347414 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.347260    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" event={"ID":"f7047780-2249-44f0-a3e4-e610c0b56f50","Type":"ContainerDied","Data":"0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787"}
Aug 04 18:23:10.349010 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.348983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" event={"ID":"a240ae88-abdc-47c9-977c-f6b3fd943a01","Type":"ContainerStarted","Data":"9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b"}
Aug 04 18:23:10.349147 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.349034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" event={"ID":"a240ae88-abdc-47c9-977c-f6b3fd943a01","Type":"ContainerStarted","Data":"18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569"}
Aug 04 18:23:10.349147 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.349063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" event={"ID":"a240ae88-abdc-47c9-977c-f6b3fd943a01","Type":"ContainerStarted","Data":"006f5dcb2317c7e85e73ab16c48047a0e99120af217ea6663bfc5f8199aec824"}
Aug 04 18:23:10.349253 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.349167    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:10.349253 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.349211    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:10.350549 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.350523    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 04 18:23:10.367372 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:10.367316    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podStartSLOduration=1.367297357 podStartE2EDuration="1.367297357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:10.366273736 +0000 UTC m=+1350.048722056" watchObservedRunningTime="2026-08-04 18:23:10.367297357 +0000 UTC m=+1350.049745683"
Aug 04 18:23:11.354146 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:11.354104    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 04 18:23:12.993413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:12.993366    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Aug 04 18:23:12.999148 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:12.999108    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 04 18:23:13.181470 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.181437    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:23:13.278547 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.278484    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-rbac-proxy-sar-config\") pod \"f7047780-2249-44f0-a3e4-e610c0b56f50\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") "
Aug 04 18:23:13.278741 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.278569    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gtd6\" (UniqueName: \"kubernetes.io/projected/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-api-access-4gtd6\") pod \"f7047780-2249-44f0-a3e4-e610c0b56f50\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") "
Aug 04 18:23:13.278741 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.278625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7047780-2249-44f0-a3e4-e610c0b56f50-proxy-tls\") pod \"f7047780-2249-44f0-a3e4-e610c0b56f50\" (UID: \"f7047780-2249-44f0-a3e4-e610c0b56f50\") "
Aug 04 18:23:13.278873 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.278848    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f7047780-2249-44f0-a3e4-e610c0b56f50" (UID: "f7047780-2249-44f0-a3e4-e610c0b56f50"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:13.280683 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.280647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7047780-2249-44f0-a3e4-e610c0b56f50-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f7047780-2249-44f0-a3e4-e610c0b56f50" (UID: "f7047780-2249-44f0-a3e4-e610c0b56f50"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:23:13.280810 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.280770    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-api-access-4gtd6" (OuterVolumeSpecName: "kube-api-access-4gtd6") pod "f7047780-2249-44f0-a3e4-e610c0b56f50" (UID: "f7047780-2249-44f0-a3e4-e610c0b56f50"). InnerVolumeSpecName "kube-api-access-4gtd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:13.364077 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.364040    2562 generic.go:356] "Generic (PLEG): container finished" podID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerID="10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2" exitCode=0
Aug 04 18:23:13.364254 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.364086    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" event={"ID":"f7047780-2249-44f0-a3e4-e610c0b56f50","Type":"ContainerDied","Data":"10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2"}
Aug 04 18:23:13.364254 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.364131    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd" event={"ID":"f7047780-2249-44f0-a3e4-e610c0b56f50","Type":"ContainerDied","Data":"dd5e8ee2df635dd0460e5a98d4634af9a491f6ccb151a1ff08fc411557419943"}
Aug 04 18:23:13.364254 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.364088    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"
Aug 04 18:23:13.364254 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.364158    2562 scope.go:117] "RemoveContainer" containerID="0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787"
Aug 04 18:23:13.373078 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.373052    2562 scope.go:117] "RemoveContainer" containerID="10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2"
Aug 04 18:23:13.379234 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.379195    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.379234 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.379219    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gtd6\" (UniqueName: \"kubernetes.io/projected/f7047780-2249-44f0-a3e4-e610c0b56f50-kube-api-access-4gtd6\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.379234 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.379229    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7047780-2249-44f0-a3e4-e610c0b56f50-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.381010 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.380989    2562 scope.go:117] "RemoveContainer" containerID="0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787"
Aug 04 18:23:13.381328 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:23:13.381311    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787\": container with ID starting with 0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787 not found: ID does not exist" containerID="0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787"
Aug 04 18:23:13.381376 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.381338    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787"} err="failed to get container status \"0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787\": rpc error: code = NotFound desc = could not find container \"0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787\": container with ID starting with 0bda13b669bc8d265837d95f720022a6b0a6ab5826d75ce426ed18320048d787 not found: ID does not exist"
Aug 04 18:23:13.381376 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.381357    2562 scope.go:117] "RemoveContainer" containerID="10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2"
Aug 04 18:23:13.381634 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:23:13.381614    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2\": container with ID starting with 10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2 not found: ID does not exist" containerID="10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2"
Aug 04 18:23:13.381696 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.381639    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2"} err="failed to get container status \"10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2\": rpc error: code = NotFound desc = could not find container \"10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2\": container with ID starting with 10889e35f9cccac0d4a4e26cb1eea061e62002951a135879391a4966b1a774d2 not found: ID does not exist"
Aug 04 18:23:13.387008 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.386979    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"]
Aug 04 18:23:13.391269 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:13.391221    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-315c2-predictor-f5d6df9c5-tb8xd"]
Aug 04 18:23:14.807773 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:14.807736    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" path="/var/lib/kubelet/pods/f7047780-2249-44f0-a3e4-e610c0b56f50/volumes"
Aug 04 18:23:16.358920 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:16.358889    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:23:16.359376 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:16.359346    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 04 18:23:19.985711 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:19.985668    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"]
Aug 04 18:23:19.986141 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:19.986022    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container" containerID="cri-o://81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e" gracePeriod=30
Aug 04 18:23:19.986141 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:19.986061    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kube-rbac-proxy" containerID="cri-o://e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c" gracePeriod=30
Aug 04 18:23:20.165953 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.165918    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"]
Aug 04 18:23:20.166422 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.166407    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kube-rbac-proxy"
Aug 04 18:23:20.166478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.166424    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kube-rbac-proxy"
Aug 04 18:23:20.166478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.166443    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container"
Aug 04 18:23:20.166478 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.166452    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container"
Aug 04 18:23:20.166651 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.166568    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kube-rbac-proxy"
Aug 04 18:23:20.166651 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.166582    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7047780-2249-44f0-a3e4-e610c0b56f50" containerName="kserve-container"
Aug 04 18:23:20.169733 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.169710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.172413 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.172386    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a162f-predictor-serving-cert\""
Aug 04 18:23:20.172628 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.172420    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a162f-kube-rbac-proxy-sar-config\""
Aug 04 18:23:20.180278 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.180244    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"]
Aug 04 18:23:20.236791 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.236682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.236791 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.236772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.237123 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.236800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkqj4\" (UniqueName: \"kubernetes.io/projected/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-api-access-mkqj4\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.337360 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.337308    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.337578 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.337382    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.337578 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.337405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkqj4\" (UniqueName: \"kubernetes.io/projected/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-api-access-mkqj4\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.337578 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:23:20.337535    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a162f-predictor-serving-cert: secret "error-404-isvc-a162f-predictor-serving-cert" not found
Aug 04 18:23:20.337762 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:23:20.337611    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls podName:49a931de-9b87-4ebd-a2f2-5449562aa016 nodeName:}" failed. No retries permitted until 2026-08-04 18:23:20.837588341 +0000 UTC m=+1360.520036656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls") pod "error-404-isvc-a162f-predictor-56c4c966dc-db74g" (UID: "49a931de-9b87-4ebd-a2f2-5449562aa016") : secret "error-404-isvc-a162f-predictor-serving-cert" not found
Aug 04 18:23:20.338054 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.338035    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.349512 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.349472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkqj4\" (UniqueName: \"kubernetes.io/projected/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-api-access-mkqj4\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.403432 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.403393    2562 generic.go:356] "Generic (PLEG): container finished" podID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerID="e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c" exitCode=2
Aug 04 18:23:20.403618 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.403440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" event={"ID":"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41","Type":"ContainerDied","Data":"e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c"}
Aug 04 18:23:20.841513 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.841453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:20.844059 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:20.844019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls\") pod \"error-404-isvc-a162f-predictor-56c4c966dc-db74g\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") " pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:21.081804 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:21.081754    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:21.234066 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:21.234039    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"]
Aug 04 18:23:21.236564 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:23:21.236535    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a931de_9b87_4ebd_a2f2_5449562aa016.slice/crio-81d06d4d19e63c67ac9c7f251f0c9a853bacb7c205dd1eac15120d763378bd0f WatchSource:0}: Error finding container 81d06d4d19e63c67ac9c7f251f0c9a853bacb7c205dd1eac15120d763378bd0f: Status 404 returned error can't find the container with id 81d06d4d19e63c67ac9c7f251f0c9a853bacb7c205dd1eac15120d763378bd0f
Aug 04 18:23:21.409323 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:21.409285    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" event={"ID":"49a931de-9b87-4ebd-a2f2-5449562aa016","Type":"ContainerStarted","Data":"24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34"}
Aug 04 18:23:21.409511 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:21.409347    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" event={"ID":"49a931de-9b87-4ebd-a2f2-5449562aa016","Type":"ContainerStarted","Data":"d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799"}
Aug 04 18:23:21.409511 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:21.409378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" event={"ID":"49a931de-9b87-4ebd-a2f2-5449562aa016","Type":"ContainerStarted","Data":"81d06d4d19e63c67ac9c7f251f0c9a853bacb7c205dd1eac15120d763378bd0f"}
Aug 04 18:23:21.409511 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:21.409408    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:21.430686 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:21.430628    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podStartSLOduration=1.4306106650000001 podStartE2EDuration="1.430610665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:21.429662112 +0000 UTC m=+1361.112110429" watchObservedRunningTime="2026-08-04 18:23:21.430610665 +0000 UTC m=+1361.113058983"
Aug 04 18:23:22.413520 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:22.413482    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:22.414995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:22.414960    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 04 18:23:23.095228 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.095179    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Aug 04 18:23:23.418247 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.418155    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 04 18:23:23.750007 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.749978    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:23:23.762311 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.762273    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlvkm\" (UniqueName: \"kubernetes.io/projected/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-api-access-mlvkm\") pod \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") "
Aug 04 18:23:23.762311 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.762314    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-proxy-tls\") pod \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") "
Aug 04 18:23:23.762559 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.762341    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-rbac-proxy-sar-config\") pod \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\" (UID: \"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41\") "
Aug 04 18:23:23.762754 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.762725    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" (UID: "c6c872a0-542c-4a67-9fa8-d9a4e5c69e41"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:23.764351 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.764323    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" (UID: "c6c872a0-542c-4a67-9fa8-d9a4e5c69e41"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:23:23.764473 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.764453    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-api-access-mlvkm" (OuterVolumeSpecName: "kube-api-access-mlvkm") pod "c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" (UID: "c6c872a0-542c-4a67-9fa8-d9a4e5c69e41"). InnerVolumeSpecName "kube-api-access-mlvkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:23.863322 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.863280    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:23:23.863322 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.863316    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:23:23.863322 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:23.863328    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlvkm\" (UniqueName: \"kubernetes.io/projected/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41-kube-api-access-mlvkm\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:23:24.423432 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.423390    2562 generic.go:356] "Generic (PLEG): container finished" podID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerID="81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e" exitCode=0
Aug 04 18:23:24.423850 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.423443    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"
Aug 04 18:23:24.423850 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.423474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" event={"ID":"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41","Type":"ContainerDied","Data":"81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e"}
Aug 04 18:23:24.423850 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.423587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m" event={"ID":"c6c872a0-542c-4a67-9fa8-d9a4e5c69e41","Type":"ContainerDied","Data":"a7a11a417f152742835cdf7f7e6b5d9c631f690ba8dc5d6031ae3e12a21d0ee8"}
Aug 04 18:23:24.423850 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.423616    2562 scope.go:117] "RemoveContainer" containerID="e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c"
Aug 04 18:23:24.432607 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.432578    2562 scope.go:117] "RemoveContainer" containerID="81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e"
Aug 04 18:23:24.442602 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.442574    2562 scope.go:117] "RemoveContainer" containerID="e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c"
Aug 04 18:23:24.442900 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:23:24.442880    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c\": container with ID starting with e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c not found: ID does not exist" containerID="e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c"
Aug 04 18:23:24.442958 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.442911    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c"} err="failed to get container status \"e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c\": rpc error: code = NotFound desc = could not find container \"e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c\": container with ID starting with e19bc97880ba7602581f9ab3f6ef24e6b7abb3df2a1ec8b8ca1753662597699c not found: ID does not exist"
Aug 04 18:23:24.442958 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.442930    2562 scope.go:117] "RemoveContainer" containerID="81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e"
Aug 04 18:23:24.443236 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:23:24.443214    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e\": container with ID starting with 81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e not found: ID does not exist" containerID="81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e"
Aug 04 18:23:24.443292 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.443244    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e"} err="failed to get container status \"81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e\": rpc error: code = NotFound desc = could not find container \"81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e\": container with ID starting with 81200190aadc1f33d66a3677f35cd8ffd5bd5090b0dac3392bd25b380f81ed0e not found: ID does not exist"
Aug 04 18:23:24.447947 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.447919    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"]
Aug 04 18:23:24.453199 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.453171    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1525-predictor-7f846d5f79-7nw6m"]
Aug 04 18:23:24.807769 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:24.807731    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" path="/var/lib/kubelet/pods/c6c872a0-542c-4a67-9fa8-d9a4e5c69e41/volumes"
Aug 04 18:23:26.359843 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:26.359795    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 04 18:23:28.422715 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:28.422682    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:23:28.423304 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:28.423270    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 04 18:23:36.360268 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:36.360222    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 04 18:23:38.423363 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:38.423318    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 04 18:23:46.359426 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:46.359380    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 04 18:23:48.423486 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:48.423445    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 04 18:23:56.359613 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:56.359561    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 04 18:23:58.423534 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:23:58.423456    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 04 18:24:06.360672 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:06.360640    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:24:08.424488 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:08.424455    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:24:20.007079 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.007038    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"]
Aug 04 18:24:20.007597 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.007393    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container" containerID="cri-o://18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569" gracePeriod=30
Aug 04 18:24:20.007597 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.007412    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kube-rbac-proxy" containerID="cri-o://9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b" gracePeriod=30
Aug 04 18:24:20.387814 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.387713    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"]
Aug 04 18:24:20.388099 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.388085    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kube-rbac-proxy"
Aug 04 18:24:20.388161 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.388100    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kube-rbac-proxy"
Aug 04 18:24:20.388161 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.388112    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container"
Aug 04 18:24:20.388161 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.388117    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container"
Aug 04 18:24:20.388309 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.388180    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kube-rbac-proxy"
Aug 04 18:24:20.388309 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.388196    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6c872a0-542c-4a67-9fa8-d9a4e5c69e41" containerName="kserve-container"
Aug 04 18:24:20.391371 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.391350    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.396279 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.396254    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d5455-predictor-serving-cert\""
Aug 04 18:24:20.397005 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.396977    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d5455-kube-rbac-proxy-sar-config\""
Aug 04 18:24:20.418397 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.418359    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"]
Aug 04 18:24:20.525242 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.525200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0364c50a-ef1e-4506-9e9a-fdbf566766e4-proxy-tls\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.525425 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.525252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.525425 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.525336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd6xt\" (UniqueName: \"kubernetes.io/projected/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-api-access-zd6xt\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.625813 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.625771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd6xt\" (UniqueName: \"kubernetes.io/projected/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-api-access-zd6xt\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.626026 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.625852    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0364c50a-ef1e-4506-9e9a-fdbf566766e4-proxy-tls\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.626026 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.625890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.626515 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.626477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.628427 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.628396    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0364c50a-ef1e-4506-9e9a-fdbf566766e4-proxy-tls\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.641122 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.641047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd6xt\" (UniqueName: \"kubernetes.io/projected/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-api-access-zd6xt\") pod \"error-404-isvc-d5455-predictor-788d7ccf85-mskgp\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") " pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.677611 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.677572    2562 generic.go:356] "Generic (PLEG): container finished" podID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerID="9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b" exitCode=2
Aug 04 18:24:20.677780 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.677632    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" event={"ID":"a240ae88-abdc-47c9-977c-f6b3fd943a01","Type":"ContainerDied","Data":"9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b"}
Aug 04 18:24:20.705825 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.705795    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:20.859432 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:20.859393    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"]
Aug 04 18:24:20.862647 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:24:20.862617    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0364c50a_ef1e_4506_9e9a_fdbf566766e4.slice/crio-9b3e47add964ca410c68994007b1905f5059508a76b00d1c0689234f80047c46 WatchSource:0}: Error finding container 9b3e47add964ca410c68994007b1905f5059508a76b00d1c0689234f80047c46: Status 404 returned error can't find the container with id 9b3e47add964ca410c68994007b1905f5059508a76b00d1c0689234f80047c46
Aug 04 18:24:21.355051 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.354996    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Aug 04 18:24:21.685363 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.685266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" event={"ID":"0364c50a-ef1e-4506-9e9a-fdbf566766e4","Type":"ContainerStarted","Data":"95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3"}
Aug 04 18:24:21.685363 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.685325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" event={"ID":"0364c50a-ef1e-4506-9e9a-fdbf566766e4","Type":"ContainerStarted","Data":"4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b"}
Aug 04 18:24:21.685363 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.685353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" event={"ID":"0364c50a-ef1e-4506-9e9a-fdbf566766e4","Type":"ContainerStarted","Data":"9b3e47add964ca410c68994007b1905f5059508a76b00d1c0689234f80047c46"}
Aug 04 18:24:21.685668 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.685379    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:21.685668 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.685425    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:21.687083 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.687053    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Aug 04 18:24:21.717270 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:21.717213    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podStartSLOduration=1.717194409 podStartE2EDuration="1.717194409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:24:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:24:21.714995437 +0000 UTC m=+1421.397443755" watchObservedRunningTime="2026-08-04 18:24:21.717194409 +0000 UTC m=+1421.399642726"
Aug 04 18:24:22.695225 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:22.695146    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Aug 04 18:24:23.668517 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.668483    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:24:23.701529 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.701426    2562 generic.go:356] "Generic (PLEG): container finished" podID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerID="18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569" exitCode=0
Aug 04 18:24:23.701529 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.701475    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"
Aug 04 18:24:23.701529 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.701494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" event={"ID":"a240ae88-abdc-47c9-977c-f6b3fd943a01","Type":"ContainerDied","Data":"18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569"}
Aug 04 18:24:23.702013 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.701578    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9" event={"ID":"a240ae88-abdc-47c9-977c-f6b3fd943a01","Type":"ContainerDied","Data":"006f5dcb2317c7e85e73ab16c48047a0e99120af217ea6663bfc5f8199aec824"}
Aug 04 18:24:23.702013 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.701607    2562 scope.go:117] "RemoveContainer" containerID="9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b"
Aug 04 18:24:23.710005 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.709980    2562 scope.go:117] "RemoveContainer" containerID="18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569"
Aug 04 18:24:23.718192 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.718166    2562 scope.go:117] "RemoveContainer" containerID="9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b"
Aug 04 18:24:23.718576 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:24:23.718545    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b\": container with ID starting with 9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b not found: ID does not exist" containerID="9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b"
Aug 04 18:24:23.718707 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.718586    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b"} err="failed to get container status \"9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b\": rpc error: code = NotFound desc = could not find container \"9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b\": container with ID starting with 9b01d15657e83af0e803d82fa819eaacd886315fea01f7f7d4cc3a07e3b5197b not found: ID does not exist"
Aug 04 18:24:23.718707 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.718607    2562 scope.go:117] "RemoveContainer" containerID="18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569"
Aug 04 18:24:23.718898 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:24:23.718874    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569\": container with ID starting with 18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569 not found: ID does not exist" containerID="18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569"
Aug 04 18:24:23.718972 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.718905    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569"} err="failed to get container status \"18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569\": rpc error: code = NotFound desc = could not find container \"18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569\": container with ID starting with 18e75dbc2ca55af8a9c15074e45f7d63da0aa6a82773abb7fdf39c0bd1d44569 not found: ID does not exist"
Aug 04 18:24:23.852520 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.852456    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a240ae88-abdc-47c9-977c-f6b3fd943a01-proxy-tls\") pod \"a240ae88-abdc-47c9-977c-f6b3fd943a01\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") "
Aug 04 18:24:23.852699 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.852562    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-rbac-proxy-sar-config\") pod \"a240ae88-abdc-47c9-977c-f6b3fd943a01\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") "
Aug 04 18:24:23.852699 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.852592    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnntp\" (UniqueName: \"kubernetes.io/projected/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-api-access-lnntp\") pod \"a240ae88-abdc-47c9-977c-f6b3fd943a01\" (UID: \"a240ae88-abdc-47c9-977c-f6b3fd943a01\") "
Aug 04 18:24:23.852978 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.852953    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a240ae88-abdc-47c9-977c-f6b3fd943a01" (UID: "a240ae88-abdc-47c9-977c-f6b3fd943a01"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:24:23.854715 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.854682    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a240ae88-abdc-47c9-977c-f6b3fd943a01-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a240ae88-abdc-47c9-977c-f6b3fd943a01" (UID: "a240ae88-abdc-47c9-977c-f6b3fd943a01"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:24:23.854815 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.854719    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-api-access-lnntp" (OuterVolumeSpecName: "kube-api-access-lnntp") pod "a240ae88-abdc-47c9-977c-f6b3fd943a01" (UID: "a240ae88-abdc-47c9-977c-f6b3fd943a01"). InnerVolumeSpecName "kube-api-access-lnntp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:24:23.953930 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.953833    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:24:23.953930 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.953866    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnntp\" (UniqueName: \"kubernetes.io/projected/a240ae88-abdc-47c9-977c-f6b3fd943a01-kube-api-access-lnntp\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:24:23.953930 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:23.953876    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a240ae88-abdc-47c9-977c-f6b3fd943a01-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:24:24.026994 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:24.026956    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"]
Aug 04 18:24:24.032703 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:24.032648    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-94d00-predictor-558f596f5c-5fxt9"]
Aug 04 18:24:24.808802 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:24.808765    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" path="/var/lib/kubelet/pods/a240ae88-abdc-47c9-977c-f6b3fd943a01/volumes"
Aug 04 18:24:27.699561 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:27.699516    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:24:27.700175 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:27.700146    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Aug 04 18:24:37.700817 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:37.700719    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Aug 04 18:24:47.700647 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:47.700607    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Aug 04 18:24:57.701064 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:24:57.701020    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Aug 04 18:25:07.701531 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:25:07.701479    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:32:35.234031 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.233989    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"]
Aug 04 18:32:35.236679 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.234700    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container"
Aug 04 18:32:35.236679 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.234721    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container"
Aug 04 18:32:35.236679 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.234741    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kube-rbac-proxy"
Aug 04 18:32:35.236679 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.234749    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kube-rbac-proxy"
Aug 04 18:32:35.236679 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.234843    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kube-rbac-proxy"
Aug 04 18:32:35.236679 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.234859    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a240ae88-abdc-47c9-977c-f6b3fd943a01" containerName="kserve-container"
Aug 04 18:32:35.237873 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.237855    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.238593 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.238566    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"]
Aug 04 18:32:35.238870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.238815    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" containerID="cri-o://d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799" gracePeriod=30
Aug 04 18:32:35.238974 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.238891    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kube-rbac-proxy" containerID="cri-o://24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34" gracePeriod=30
Aug 04 18:32:35.241793 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.241717    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8c81a-kube-rbac-proxy-sar-config\""
Aug 04 18:32:35.241793 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.241717    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8c81a-predictor-serving-cert\""
Aug 04 18:32:35.256896 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.256866    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"]
Aug 04 18:32:35.304900 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.304865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.305082 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.304937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9kqx\" (UniqueName: \"kubernetes.io/projected/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-api-access-c9kqx\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.305082 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.304988    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-proxy-tls\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.405573 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.405538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9kqx\" (UniqueName: \"kubernetes.io/projected/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-api-access-c9kqx\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.405573 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.405574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-proxy-tls\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.405795 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.405626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.406333 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.406308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.408073 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.408050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-proxy-tls\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.420577 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.420543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9kqx\" (UniqueName: \"kubernetes.io/projected/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-api-access-c9kqx\") pod \"error-404-isvc-8c81a-predictor-756657c85-2pmpb\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") " pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.558691 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.558645    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:35.712335 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.712305    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"]
Aug 04 18:32:35.713068 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:32:35.713034    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de5a5a5_de93_459b_8b6c_89d493d3e1f9.slice/crio-a6ec6b17a432697081108ff9f4e53dd34b1d6ae8164cff23a3ec7ecd8cb71ea4 WatchSource:0}: Error finding container a6ec6b17a432697081108ff9f4e53dd34b1d6ae8164cff23a3ec7ecd8cb71ea4: Status 404 returned error can't find the container with id a6ec6b17a432697081108ff9f4e53dd34b1d6ae8164cff23a3ec7ecd8cb71ea4
Aug 04 18:32:35.736812 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.736783    2562 generic.go:356] "Generic (PLEG): container finished" podID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerID="24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34" exitCode=2
Aug 04 18:32:35.736955 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.736862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" event={"ID":"49a931de-9b87-4ebd-a2f2-5449562aa016","Type":"ContainerDied","Data":"24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34"}
Aug 04 18:32:35.738003 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:35.737982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" event={"ID":"0de5a5a5-de93-459b-8b6c-89d493d3e1f9","Type":"ContainerStarted","Data":"a6ec6b17a432697081108ff9f4e53dd34b1d6ae8164cff23a3ec7ecd8cb71ea4"}
Aug 04 18:32:36.744818 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:36.744779    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" event={"ID":"0de5a5a5-de93-459b-8b6c-89d493d3e1f9","Type":"ContainerStarted","Data":"73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3"}
Aug 04 18:32:36.745258 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:36.744838    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" event={"ID":"0de5a5a5-de93-459b-8b6c-89d493d3e1f9","Type":"ContainerStarted","Data":"349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03"}
Aug 04 18:32:36.745258 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:36.744865    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:36.745258 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:36.744886    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:36.746191 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:36.746162    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Aug 04 18:32:36.769078 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:36.769026    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podStartSLOduration=1.769007867 podStartE2EDuration="1.769007867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:32:36.767072095 +0000 UTC m=+1916.449520414" watchObservedRunningTime="2026-08-04 18:32:36.769007867 +0000 UTC m=+1916.451456257"
Aug 04 18:32:37.748989 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:37.748950    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Aug 04 18:32:38.419234 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.419192    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Aug 04 18:32:38.423551 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.423521    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 04 18:32:38.591856 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.591826    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:32:38.627152 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.627122    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkqj4\" (UniqueName: \"kubernetes.io/projected/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-api-access-mkqj4\") pod \"49a931de-9b87-4ebd-a2f2-5449562aa016\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") "
Aug 04 18:32:38.627337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.627217    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-rbac-proxy-sar-config\") pod \"49a931de-9b87-4ebd-a2f2-5449562aa016\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") "
Aug 04 18:32:38.627337 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.627235    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls\") pod \"49a931de-9b87-4ebd-a2f2-5449562aa016\" (UID: \"49a931de-9b87-4ebd-a2f2-5449562aa016\") "
Aug 04 18:32:38.627719 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.627682    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "49a931de-9b87-4ebd-a2f2-5449562aa016" (UID: "49a931de-9b87-4ebd-a2f2-5449562aa016"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:32:38.629288 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.629259    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-api-access-mkqj4" (OuterVolumeSpecName: "kube-api-access-mkqj4") pod "49a931de-9b87-4ebd-a2f2-5449562aa016" (UID: "49a931de-9b87-4ebd-a2f2-5449562aa016"). InnerVolumeSpecName "kube-api-access-mkqj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:32:38.629384 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.629341    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "49a931de-9b87-4ebd-a2f2-5449562aa016" (UID: "49a931de-9b87-4ebd-a2f2-5449562aa016"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:32:38.728096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.727997    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:32:38.728096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.728043    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49a931de-9b87-4ebd-a2f2-5449562aa016-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:32:38.728096 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.728054    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkqj4\" (UniqueName: \"kubernetes.io/projected/49a931de-9b87-4ebd-a2f2-5449562aa016-kube-api-access-mkqj4\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:32:38.754093 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.754060    2562 generic.go:356] "Generic (PLEG): container finished" podID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerID="d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799" exitCode=0
Aug 04 18:32:38.754547 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.754114    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"
Aug 04 18:32:38.754547 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.754141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" event={"ID":"49a931de-9b87-4ebd-a2f2-5449562aa016","Type":"ContainerDied","Data":"d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799"}
Aug 04 18:32:38.754547 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.754204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g" event={"ID":"49a931de-9b87-4ebd-a2f2-5449562aa016","Type":"ContainerDied","Data":"81d06d4d19e63c67ac9c7f251f0c9a853bacb7c205dd1eac15120d763378bd0f"}
Aug 04 18:32:38.754547 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.754233    2562 scope.go:117] "RemoveContainer" containerID="24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34"
Aug 04 18:32:38.764183 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.764158    2562 scope.go:117] "RemoveContainer" containerID="d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799"
Aug 04 18:32:38.772870 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.772851    2562 scope.go:117] "RemoveContainer" containerID="24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34"
Aug 04 18:32:38.773154 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:32:38.773127    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34\": container with ID starting with 24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34 not found: ID does not exist" containerID="24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34"
Aug 04 18:32:38.773212 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.773167    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34"} err="failed to get container status \"24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34\": rpc error: code = NotFound desc = could not find container \"24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34\": container with ID starting with 24d549e902aa79bbfdad6001baa4de648f93b47acbef5bdf45dae82ff5509d34 not found: ID does not exist"
Aug 04 18:32:38.773212 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.773194    2562 scope.go:117] "RemoveContainer" containerID="d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799"
Aug 04 18:32:38.773484 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:32:38.773459    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799\": container with ID starting with d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799 not found: ID does not exist" containerID="d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799"
Aug 04 18:32:38.773570 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.773496    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799"} err="failed to get container status \"d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799\": rpc error: code = NotFound desc = could not find container \"d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799\": container with ID starting with d656c6a5dd3eb2245c6e32ed32c43e0ce49f4f2166cd39abe964fca4208cb799 not found: ID does not exist"
Aug 04 18:32:38.785475 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.785448    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"]
Aug 04 18:32:38.790720 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.790697    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a162f-predictor-56c4c966dc-db74g"]
Aug 04 18:32:38.807999 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:38.807972    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" path="/var/lib/kubelet/pods/49a931de-9b87-4ebd-a2f2-5449562aa016/volumes"
Aug 04 18:32:42.753759 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:42.753728    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:32:42.754270 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:42.754242    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Aug 04 18:32:52.755147 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:32:52.755104    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Aug 04 18:33:02.755080 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:02.755036    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Aug 04 18:33:12.754851 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:12.754803    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Aug 04 18:33:22.755688 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:22.755652    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:33:35.018423 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.018345    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"]
Aug 04 18:33:35.018962 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.018582    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" containerID="cri-o://4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b" gracePeriod=30
Aug 04 18:33:35.018962 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.018637    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kube-rbac-proxy" containerID="cri-o://95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3" gracePeriod=30
Aug 04 18:33:35.145028 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.144997    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"]
Aug 04 18:33:35.145482 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.145467    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container"
Aug 04 18:33:35.145542 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.145486    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container"
Aug 04 18:33:35.145579 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.145540    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kube-rbac-proxy"
Aug 04 18:33:35.145579 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.145550    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kube-rbac-proxy"
Aug 04 18:33:35.145661 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.145650    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kserve-container"
Aug 04 18:33:35.145693 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.145668    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="49a931de-9b87-4ebd-a2f2-5449562aa016" containerName="kube-rbac-proxy"
Aug 04 18:33:35.156250 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.156204    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.158872 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.158851    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-493ba-predictor-serving-cert\""
Aug 04 18:33:35.159070 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.159046    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-493ba-kube-rbac-proxy-sar-config\""
Aug 04 18:33:35.159359 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.159337    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"]
Aug 04 18:33:35.178114 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.178085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f87730bd-b669-4029-a664-907e762cdba6-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.178291 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.178120    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.178291 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.178268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp6tf\" (UniqueName: \"kubernetes.io/projected/f87730bd-b669-4029-a664-907e762cdba6-kube-api-access-qp6tf\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.278788 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.278689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp6tf\" (UniqueName: \"kubernetes.io/projected/f87730bd-b669-4029-a664-907e762cdba6-kube-api-access-qp6tf\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.278788 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.278775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f87730bd-b669-4029-a664-907e762cdba6-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.278988 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.278803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.278988 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:33:35.278927    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-493ba-predictor-serving-cert: secret "error-404-isvc-493ba-predictor-serving-cert" not found
Aug 04 18:33:35.279074 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:33:35.279001    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls podName:f87730bd-b669-4029-a664-907e762cdba6 nodeName:}" failed. No retries permitted until 2026-08-04 18:33:35.778981679 +0000 UTC m=+1975.461429974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls") pod "error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" (UID: "f87730bd-b669-4029-a664-907e762cdba6") : secret "error-404-isvc-493ba-predictor-serving-cert" not found
Aug 04 18:33:35.279473 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.279453    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f87730bd-b669-4029-a664-907e762cdba6-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.288484 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.288455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp6tf\" (UniqueName: \"kubernetes.io/projected/f87730bd-b669-4029-a664-907e762cdba6-kube-api-access-qp6tf\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.783897 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.783859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:35.786365 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:35.786339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls\") pod \"error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") " pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:36.009997 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:36.009958    2562 generic.go:356] "Generic (PLEG): container finished" podID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerID="95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3" exitCode=2
Aug 04 18:33:36.010196 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:36.010011    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" event={"ID":"0364c50a-ef1e-4506-9e9a-fdbf566766e4","Type":"ContainerDied","Data":"95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3"}
Aug 04 18:33:36.069365 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:36.069270    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:36.210541 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:36.210515    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"]
Aug 04 18:33:36.212825 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:33:36.212793    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87730bd_b669_4029_a664_907e762cdba6.slice/crio-7c1e4da6b08f75d79316fc4baee6f96dac4cb7288cf53cb3926eba63b6a7eb08 WatchSource:0}: Error finding container 7c1e4da6b08f75d79316fc4baee6f96dac4cb7288cf53cb3926eba63b6a7eb08: Status 404 returned error can't find the container with id 7c1e4da6b08f75d79316fc4baee6f96dac4cb7288cf53cb3926eba63b6a7eb08
Aug 04 18:33:37.016244 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:37.016204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" event={"ID":"f87730bd-b669-4029-a664-907e762cdba6","Type":"ContainerStarted","Data":"bf8d278ab8dea5552773a0043c6adb816f1ad1b63458c7e8e119328ee3fcde83"}
Aug 04 18:33:37.016553 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:37.016277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" event={"ID":"f87730bd-b669-4029-a664-907e762cdba6","Type":"ContainerStarted","Data":"9f9e13e2843bc9e0d7ef5230a8840e23103dde4811c87a0e72aa8af9cc0972e1"}
Aug 04 18:33:37.016553 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:37.016312    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:37.016553 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:37.016338    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" event={"ID":"f87730bd-b669-4029-a664-907e762cdba6","Type":"ContainerStarted","Data":"7c1e4da6b08f75d79316fc4baee6f96dac4cb7288cf53cb3926eba63b6a7eb08"}
Aug 04 18:33:37.044647 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:37.044591    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podStartSLOduration=2.044573132 podStartE2EDuration="2.044573132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:37.042721099 +0000 UTC m=+1976.725169417" watchObservedRunningTime="2026-08-04 18:33:37.044573132 +0000 UTC m=+1976.727021450"
Aug 04 18:33:37.695752 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:37.695706    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": dial tcp 10.133.0.32:8643: connect: connection refused"
Aug 04 18:33:37.700673 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:37.700640    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Aug 04 18:33:38.020426 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.020389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:38.021674 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.021644    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Aug 04 18:33:38.482633 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.482611    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:33:38.506618 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.506591    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0364c50a-ef1e-4506-9e9a-fdbf566766e4-proxy-tls\") pod \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") "
Aug 04 18:33:38.506765 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.506631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd6xt\" (UniqueName: \"kubernetes.io/projected/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-api-access-zd6xt\") pod \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") "
Aug 04 18:33:38.506765 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.506673    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-rbac-proxy-sar-config\") pod \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\" (UID: \"0364c50a-ef1e-4506-9e9a-fdbf566766e4\") "
Aug 04 18:33:38.507070 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.507044    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0364c50a-ef1e-4506-9e9a-fdbf566766e4" (UID: "0364c50a-ef1e-4506-9e9a-fdbf566766e4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:33:38.508736 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.508706    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0364c50a-ef1e-4506-9e9a-fdbf566766e4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0364c50a-ef1e-4506-9e9a-fdbf566766e4" (UID: "0364c50a-ef1e-4506-9e9a-fdbf566766e4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:33:38.508874 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.508859    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-api-access-zd6xt" (OuterVolumeSpecName: "kube-api-access-zd6xt") pod "0364c50a-ef1e-4506-9e9a-fdbf566766e4" (UID: "0364c50a-ef1e-4506-9e9a-fdbf566766e4"). InnerVolumeSpecName "kube-api-access-zd6xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:33:38.607318 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.607225    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0364c50a-ef1e-4506-9e9a-fdbf566766e4-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:33:38.607318 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.607254    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd6xt\" (UniqueName: \"kubernetes.io/projected/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-api-access-zd6xt\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:33:38.607318 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:38.607265    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0364c50a-ef1e-4506-9e9a-fdbf566766e4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:33:39.026363 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.026327    2562 generic.go:356] "Generic (PLEG): container finished" podID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerID="4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b" exitCode=0
Aug 04 18:33:39.026868 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.026382    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"
Aug 04 18:33:39.026868 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.026417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" event={"ID":"0364c50a-ef1e-4506-9e9a-fdbf566766e4","Type":"ContainerDied","Data":"4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b"}
Aug 04 18:33:39.026868 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.026526    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp" event={"ID":"0364c50a-ef1e-4506-9e9a-fdbf566766e4","Type":"ContainerDied","Data":"9b3e47add964ca410c68994007b1905f5059508a76b00d1c0689234f80047c46"}
Aug 04 18:33:39.026868 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.026575    2562 scope.go:117] "RemoveContainer" containerID="95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3"
Aug 04 18:33:39.027092 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.026921    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Aug 04 18:33:39.035134 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.035112    2562 scope.go:117] "RemoveContainer" containerID="4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b"
Aug 04 18:33:39.043364 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.043337    2562 scope.go:117] "RemoveContainer" containerID="95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3"
Aug 04 18:33:39.043727 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:33:39.043703    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3\": container with ID starting with 95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3 not found: ID does not exist" containerID="95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3"
Aug 04 18:33:39.043802 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.043737    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3"} err="failed to get container status \"95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3\": rpc error: code = NotFound desc = could not find container \"95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3\": container with ID starting with 95cd6660bb5bd6bc3db1b2d09c21e105b8b032dc5f1d90d55c1ad556f3755fd3 not found: ID does not exist"
Aug 04 18:33:39.043802 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.043757    2562 scope.go:117] "RemoveContainer" containerID="4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b"
Aug 04 18:33:39.044064 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:33:39.044045    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b\": container with ID starting with 4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b not found: ID does not exist" containerID="4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b"
Aug 04 18:33:39.044124 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.044075    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b"} err="failed to get container status \"4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b\": rpc error: code = NotFound desc = could not find container \"4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b\": container with ID starting with 4b33e2adad5efe9d1de4b036b6fb694fc3db89c35d398562d11f7145f40d7e1b not found: ID does not exist"
Aug 04 18:33:39.047418 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.047389    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"]
Aug 04 18:33:39.052120 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:39.052093    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d5455-predictor-788d7ccf85-mskgp"]
Aug 04 18:33:40.808420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:40.808388    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" path="/var/lib/kubelet/pods/0364c50a-ef1e-4506-9e9a-fdbf566766e4/volumes"
Aug 04 18:33:44.031286 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:44.031257    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:33:44.031855 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:44.031825    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Aug 04 18:33:45.309305 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.309271    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"]
Aug 04 18:33:45.309735 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.309527    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container" containerID="cri-o://349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03" gracePeriod=30
Aug 04 18:33:45.309735 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.309551    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kube-rbac-proxy" containerID="cri-o://73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3" gracePeriod=30
Aug 04 18:33:45.490917 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.490884    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"]
Aug 04 18:33:45.491285 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.491268    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kube-rbac-proxy"
Aug 04 18:33:45.491354 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.491288    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kube-rbac-proxy"
Aug 04 18:33:45.491354 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.491303    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container"
Aug 04 18:33:45.491354 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.491313    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container"
Aug 04 18:33:45.491556 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.491413    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kube-rbac-proxy"
Aug 04 18:33:45.491556 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.491427    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0364c50a-ef1e-4506-9e9a-fdbf566766e4" containerName="kserve-container"
Aug 04 18:33:45.494349 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.494327    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.496893 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.496869    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-47ede-kube-rbac-proxy-sar-config\""
Aug 04 18:33:45.497191 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.497177    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-47ede-predictor-serving-cert\""
Aug 04 18:33:45.510202 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.506530    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"]
Aug 04 18:33:45.565544 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.565428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/54eba614-562f-43dd-99f8-ccf29473617b-kube-api-access-pp6pb\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.565544 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.565472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54eba614-562f-43dd-99f8-ccf29473617b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.565746 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.565556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54eba614-562f-43dd-99f8-ccf29473617b-proxy-tls\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.666097 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.666058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/54eba614-562f-43dd-99f8-ccf29473617b-kube-api-access-pp6pb\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.666097 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.666098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54eba614-562f-43dd-99f8-ccf29473617b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.666345 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.666162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54eba614-562f-43dd-99f8-ccf29473617b-proxy-tls\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.666813 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.666790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54eba614-562f-43dd-99f8-ccf29473617b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.668712 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.668690    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54eba614-562f-43dd-99f8-ccf29473617b-proxy-tls\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.675729 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.675708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/54eba614-562f-43dd-99f8-ccf29473617b-kube-api-access-pp6pb\") pod \"error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") " pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.814662 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.814627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:45.947678 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:45.947650    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"]
Aug 04 18:33:45.949711 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:33:45.949681    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54eba614_562f_43dd_99f8_ccf29473617b.slice/crio-7b4f15c40287c90e7d95574abdb7a15f19a4c6c425cbbef37605b3fa66aaa70d WatchSource:0}: Error finding container 7b4f15c40287c90e7d95574abdb7a15f19a4c6c425cbbef37605b3fa66aaa70d: Status 404 returned error can't find the container with id 7b4f15c40287c90e7d95574abdb7a15f19a4c6c425cbbef37605b3fa66aaa70d
Aug 04 18:33:46.059855 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:46.059825    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" event={"ID":"54eba614-562f-43dd-99f8-ccf29473617b","Type":"ContainerStarted","Data":"e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7"}
Aug 04 18:33:46.059975 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:46.059890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" event={"ID":"54eba614-562f-43dd-99f8-ccf29473617b","Type":"ContainerStarted","Data":"5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af"}
Aug 04 18:33:46.059975 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:46.059921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" event={"ID":"54eba614-562f-43dd-99f8-ccf29473617b","Type":"ContainerStarted","Data":"7b4f15c40287c90e7d95574abdb7a15f19a4c6c425cbbef37605b3fa66aaa70d"}
Aug 04 18:33:46.061615 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:46.061590    2562 generic.go:356] "Generic (PLEG): container finished" podID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerID="73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3" exitCode=2
Aug 04 18:33:46.061722 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:46.061668    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" event={"ID":"0de5a5a5-de93-459b-8b6c-89d493d3e1f9","Type":"ContainerDied","Data":"73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3"}
Aug 04 18:33:47.066874 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:47.066841    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:47.092420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:47.092364    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podStartSLOduration=2.092350758 podStartE2EDuration="2.092350758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:47.091162283 +0000 UTC m=+1986.773610603" watchObservedRunningTime="2026-08-04 18:33:47.092350758 +0000 UTC m=+1986.774799076"
Aug 04 18:33:47.749842 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:47.749798    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Aug 04 18:33:48.071635 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.071599    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:48.072961 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.072932    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Aug 04 18:33:48.658381 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.658355    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:33:48.691496 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.691412    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-proxy-tls\") pod \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") "
Aug 04 18:33:48.691496 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.691453    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-rbac-proxy-sar-config\") pod \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") "
Aug 04 18:33:48.691727 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.691540    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9kqx\" (UniqueName: \"kubernetes.io/projected/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-api-access-c9kqx\") pod \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\" (UID: \"0de5a5a5-de93-459b-8b6c-89d493d3e1f9\") "
Aug 04 18:33:48.691801 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.691776    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0de5a5a5-de93-459b-8b6c-89d493d3e1f9" (UID: "0de5a5a5-de93-459b-8b6c-89d493d3e1f9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:33:48.693698 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.693664    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-api-access-c9kqx" (OuterVolumeSpecName: "kube-api-access-c9kqx") pod "0de5a5a5-de93-459b-8b6c-89d493d3e1f9" (UID: "0de5a5a5-de93-459b-8b6c-89d493d3e1f9"). InnerVolumeSpecName "kube-api-access-c9kqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:33:48.693698 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.693689    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0de5a5a5-de93-459b-8b6c-89d493d3e1f9" (UID: "0de5a5a5-de93-459b-8b6c-89d493d3e1f9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:33:48.792266 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.792230    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:33:48.792266 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.792259    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:33:48.792266 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:48.792269    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9kqx\" (UniqueName: \"kubernetes.io/projected/0de5a5a5-de93-459b-8b6c-89d493d3e1f9-kube-api-access-c9kqx\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:33:49.076955 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.076918    2562 generic.go:356] "Generic (PLEG): container finished" podID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerID="349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03" exitCode=0
Aug 04 18:33:49.077456 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.076972    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"
Aug 04 18:33:49.077456 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.077003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" event={"ID":"0de5a5a5-de93-459b-8b6c-89d493d3e1f9","Type":"ContainerDied","Data":"349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03"}
Aug 04 18:33:49.077456 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.077089    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb" event={"ID":"0de5a5a5-de93-459b-8b6c-89d493d3e1f9","Type":"ContainerDied","Data":"a6ec6b17a432697081108ff9f4e53dd34b1d6ae8164cff23a3ec7ecd8cb71ea4"}
Aug 04 18:33:49.077456 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.077133    2562 scope.go:117] "RemoveContainer" containerID="73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3"
Aug 04 18:33:49.077796 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.077568    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Aug 04 18:33:49.085426 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.085413    2562 scope.go:117] "RemoveContainer" containerID="349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03"
Aug 04 18:33:49.092713 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.092693    2562 scope.go:117] "RemoveContainer" containerID="73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3"
Aug 04 18:33:49.092964 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:33:49.092944    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3\": container with ID starting with 73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3 not found: ID does not exist" containerID="73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3"
Aug 04 18:33:49.093041 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.092979    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3"} err="failed to get container status \"73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3\": rpc error: code = NotFound desc = could not find container \"73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3\": container with ID starting with 73a038c6dfe93709225c9c3cfab5f33aa6031f315a1b7d3d5047c7274ea904e3 not found: ID does not exist"
Aug 04 18:33:49.093041 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.093006    2562 scope.go:117] "RemoveContainer" containerID="349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03"
Aug 04 18:33:49.093272 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:33:49.093253    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03\": container with ID starting with 349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03 not found: ID does not exist" containerID="349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03"
Aug 04 18:33:49.093311 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.093277    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03"} err="failed to get container status \"349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03\": rpc error: code = NotFound desc = could not find container \"349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03\": container with ID starting with 349135d0735d896a908ffccd0a9004143f538f2cd67c12a2b03ae72514e84b03 not found: ID does not exist"
Aug 04 18:33:49.096339 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.096318    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"]
Aug 04 18:33:49.100880 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:49.100859    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8c81a-predictor-756657c85-2pmpb"]
Aug 04 18:33:50.809136 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:50.809106    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" path="/var/lib/kubelet/pods/0de5a5a5-de93-459b-8b6c-89d493d3e1f9/volumes"
Aug 04 18:33:54.032088 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:54.032045    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Aug 04 18:33:54.082090 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:54.082059    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:33:54.082538 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:33:54.082486    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Aug 04 18:34:04.032749 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:34:04.032699    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Aug 04 18:34:04.082960 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:34:04.082919    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Aug 04 18:34:14.032274 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:34:14.032214    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Aug 04 18:34:14.083342 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:34:14.083304    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Aug 04 18:34:24.033263 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:34:24.033221    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:34:24.082572 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:34:24.082534    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Aug 04 18:34:34.083647 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:34:34.083613    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:43:00.471193 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:00.471158    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"]
Aug 04 18:43:00.471717 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:00.471416    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container" containerID="cri-o://5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af" gracePeriod=30
Aug 04 18:43:00.471717 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:00.471473    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kube-rbac-proxy" containerID="cri-o://e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7" gracePeriod=30
Aug 04 18:43:01.397833 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:01.397797    2562 generic.go:356] "Generic (PLEG): container finished" podID="54eba614-562f-43dd-99f8-ccf29473617b" containerID="e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7" exitCode=2
Aug 04 18:43:01.398007 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:01.397871    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" event={"ID":"54eba614-562f-43dd-99f8-ccf29473617b","Type":"ContainerDied","Data":"e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7"}
Aug 04 18:43:03.716246 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.716221    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:43:03.749772 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.749745    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54eba614-562f-43dd-99f8-ccf29473617b-kube-rbac-proxy-sar-config\") pod \"54eba614-562f-43dd-99f8-ccf29473617b\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") "
Aug 04 18:43:03.749903 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.749871    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54eba614-562f-43dd-99f8-ccf29473617b-proxy-tls\") pod \"54eba614-562f-43dd-99f8-ccf29473617b\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") "
Aug 04 18:43:03.749966 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.749906    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/54eba614-562f-43dd-99f8-ccf29473617b-kube-api-access-pp6pb\") pod \"54eba614-562f-43dd-99f8-ccf29473617b\" (UID: \"54eba614-562f-43dd-99f8-ccf29473617b\") "
Aug 04 18:43:03.750118 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.750090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54eba614-562f-43dd-99f8-ccf29473617b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "54eba614-562f-43dd-99f8-ccf29473617b" (UID: "54eba614-562f-43dd-99f8-ccf29473617b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:43:03.751877 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.751853    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54eba614-562f-43dd-99f8-ccf29473617b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "54eba614-562f-43dd-99f8-ccf29473617b" (UID: "54eba614-562f-43dd-99f8-ccf29473617b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:43:03.751960 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.751929    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54eba614-562f-43dd-99f8-ccf29473617b-kube-api-access-pp6pb" (OuterVolumeSpecName: "kube-api-access-pp6pb") pod "54eba614-562f-43dd-99f8-ccf29473617b" (UID: "54eba614-562f-43dd-99f8-ccf29473617b"). InnerVolumeSpecName "kube-api-access-pp6pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:43:03.850607 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.850520    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54eba614-562f-43dd-99f8-ccf29473617b-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:43:03.850607 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.850544    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/54eba614-562f-43dd-99f8-ccf29473617b-kube-api-access-pp6pb\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:43:03.850607 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:03.850554    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54eba614-562f-43dd-99f8-ccf29473617b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:43:04.412356 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.412313    2562 generic.go:356] "Generic (PLEG): container finished" podID="54eba614-562f-43dd-99f8-ccf29473617b" containerID="5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af" exitCode=0
Aug 04 18:43:04.412356 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.412354    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"
Aug 04 18:43:04.412597 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.412364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" event={"ID":"54eba614-562f-43dd-99f8-ccf29473617b","Type":"ContainerDied","Data":"5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af"}
Aug 04 18:43:04.412597 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.412419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd" event={"ID":"54eba614-562f-43dd-99f8-ccf29473617b","Type":"ContainerDied","Data":"7b4f15c40287c90e7d95574abdb7a15f19a4c6c425cbbef37605b3fa66aaa70d"}
Aug 04 18:43:04.412597 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.412446    2562 scope.go:117] "RemoveContainer" containerID="e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7"
Aug 04 18:43:04.421568 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.421547    2562 scope.go:117] "RemoveContainer" containerID="5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af"
Aug 04 18:43:04.428634 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.428614    2562 scope.go:117] "RemoveContainer" containerID="e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7"
Aug 04 18:43:04.428926 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:43:04.428904    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7\": container with ID starting with e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7 not found: ID does not exist" containerID="e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7"
Aug 04 18:43:04.429000 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.428934    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7"} err="failed to get container status \"e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7\": rpc error: code = NotFound desc = could not find container \"e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7\": container with ID starting with e5a62904c6d587a7de4687960747f5f378977225af5cfc47fbaf067a159a7ae7 not found: ID does not exist"
Aug 04 18:43:04.429000 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.428950    2562 scope.go:117] "RemoveContainer" containerID="5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af"
Aug 04 18:43:04.429176 ip-10-0-136-239 kubenswrapper[2562]: E0804 18:43:04.429162    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af\": container with ID starting with 5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af not found: ID does not exist" containerID="5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af"
Aug 04 18:43:04.429213 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.429178    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af"} err="failed to get container status \"5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af\": rpc error: code = NotFound desc = could not find container \"5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af\": container with ID starting with 5ddf491328a202520a2f35772843aea163b6ffbac4becb2e24f4ac5abb4351af not found: ID does not exist"
Aug 04 18:43:04.437200 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.437176    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"]
Aug 04 18:43:04.442134 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.442114    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-47ede-predictor-86dbbb4b45-r2rtd"]
Aug 04 18:43:04.807754 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:43:04.807723    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54eba614-562f-43dd-99f8-ccf29473617b" path="/var/lib/kubelet/pods/54eba614-562f-43dd-99f8-ccf29473617b/volumes"
Aug 04 18:50:54.625700 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:54.625670    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"]
Aug 04 18:50:54.628047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:54.625915    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container" containerID="cri-o://9f9e13e2843bc9e0d7ef5230a8840e23103dde4811c87a0e72aa8af9cc0972e1" gracePeriod=30
Aug 04 18:50:54.628047 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:54.625971    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kube-rbac-proxy" containerID="cri-o://bf8d278ab8dea5552773a0043c6adb816f1ad1b63458c7e8e119328ee3fcde83" gracePeriod=30
Aug 04 18:50:55.308025 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:55.307990    2562 generic.go:356] "Generic (PLEG): container finished" podID="f87730bd-b669-4029-a664-907e762cdba6" containerID="bf8d278ab8dea5552773a0043c6adb816f1ad1b63458c7e8e119328ee3fcde83" exitCode=2
Aug 04 18:50:55.308187 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:55.308029    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" event={"ID":"f87730bd-b669-4029-a664-907e762cdba6","Type":"ContainerDied","Data":"bf8d278ab8dea5552773a0043c6adb816f1ad1b63458c7e8e119328ee3fcde83"}
Aug 04 18:50:57.318399 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.318368    2562 generic.go:356] "Generic (PLEG): container finished" podID="f87730bd-b669-4029-a664-907e762cdba6" containerID="9f9e13e2843bc9e0d7ef5230a8840e23103dde4811c87a0e72aa8af9cc0972e1" exitCode=0
Aug 04 18:50:57.318851 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.318444    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" event={"ID":"f87730bd-b669-4029-a664-907e762cdba6","Type":"ContainerDied","Data":"9f9e13e2843bc9e0d7ef5230a8840e23103dde4811c87a0e72aa8af9cc0972e1"}
Aug 04 18:50:57.363305 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.363285    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:50:57.422912 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.422851    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls\") pod \"f87730bd-b669-4029-a664-907e762cdba6\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") "
Aug 04 18:50:57.422912 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.422888    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp6tf\" (UniqueName: \"kubernetes.io/projected/f87730bd-b669-4029-a664-907e762cdba6-kube-api-access-qp6tf\") pod \"f87730bd-b669-4029-a664-907e762cdba6\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") "
Aug 04 18:50:57.423078 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.422989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f87730bd-b669-4029-a664-907e762cdba6-kube-rbac-proxy-sar-config\") pod \"f87730bd-b669-4029-a664-907e762cdba6\" (UID: \"f87730bd-b669-4029-a664-907e762cdba6\") "
Aug 04 18:50:57.423294 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.423266    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87730bd-b669-4029-a664-907e762cdba6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f87730bd-b669-4029-a664-907e762cdba6" (UID: "f87730bd-b669-4029-a664-907e762cdba6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:50:57.424900 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.424873    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87730bd-b669-4029-a664-907e762cdba6-kube-api-access-qp6tf" (OuterVolumeSpecName: "kube-api-access-qp6tf") pod "f87730bd-b669-4029-a664-907e762cdba6" (UID: "f87730bd-b669-4029-a664-907e762cdba6"). InnerVolumeSpecName "kube-api-access-qp6tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:50:57.424967 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.424882    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f87730bd-b669-4029-a664-907e762cdba6" (UID: "f87730bd-b669-4029-a664-907e762cdba6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:50:57.523748 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.523720    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f87730bd-b669-4029-a664-907e762cdba6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:50:57.523748 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.523743    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f87730bd-b669-4029-a664-907e762cdba6-proxy-tls\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:50:57.523748 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:57.523753    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp6tf\" (UniqueName: \"kubernetes.io/projected/f87730bd-b669-4029-a664-907e762cdba6-kube-api-access-qp6tf\") on node \"ip-10-0-136-239.ec2.internal\" DevicePath \"\""
Aug 04 18:50:58.324589 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:58.324555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv" event={"ID":"f87730bd-b669-4029-a664-907e762cdba6","Type":"ContainerDied","Data":"7c1e4da6b08f75d79316fc4baee6f96dac4cb7288cf53cb3926eba63b6a7eb08"}
Aug 04 18:50:58.325104 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:58.324597    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"
Aug 04 18:50:58.325104 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:58.324622    2562 scope.go:117] "RemoveContainer" containerID="bf8d278ab8dea5552773a0043c6adb816f1ad1b63458c7e8e119328ee3fcde83"
Aug 04 18:50:58.333664 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:58.333643    2562 scope.go:117] "RemoveContainer" containerID="9f9e13e2843bc9e0d7ef5230a8840e23103dde4811c87a0e72aa8af9cc0972e1"
Aug 04 18:50:58.349795 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:58.349751    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"]
Aug 04 18:50:58.354543 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:58.354525    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-493ba-predictor-6c4f76bbb5-dg9jv"]
Aug 04 18:50:58.807732 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:50:58.807700    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f87730bd-b669-4029-a664-907e762cdba6" path="/var/lib/kubelet/pods/f87730bd-b669-4029-a664-907e762cdba6/volumes"
Aug 04 18:51:30.202731 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:30.202699    2562 ???:1] "http2: server: error reading preface from client 10.0.128.114:57832: read tcp 10.0.136.239:10250->10.0.128.114:57832: read: connection reset by peer"
Aug 04 18:51:30.212046 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:30.212019    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2qbzg_e5b59001-bd6e-4030-83f7-0d46dcc42501/global-pull-secret-syncer/0.log"
Aug 04 18:51:30.397953 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:30.397922    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jcl8j_43e87821-b4a3-4117-84cd-3f5410422a03/konnectivity-agent/0.log"
Aug 04 18:51:30.537775 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:30.537743    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-239.ec2.internal_3b6a804075f7c4a3a5c87d537eea8844/haproxy/0.log"
Aug 04 18:51:35.110512 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:35.110420    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwlh5_e20f5604-4e33-498e-8c6f-ae34999deff6/node-exporter/0.log"
Aug 04 18:51:35.136383 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:35.136356    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwlh5_e20f5604-4e33-498e-8c6f-ae34999deff6/kube-rbac-proxy/0.log"
Aug 04 18:51:35.163762 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:35.163738    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwlh5_e20f5604-4e33-498e-8c6f-ae34999deff6/init-textfile/0.log"
Aug 04 18:51:35.692572 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:35.692534    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-dgzgj_f15ed9cc-e46b-4d42-ae94-06f6bc1a9b06/prometheus-operator-admission-webhook/0.log"
Aug 04 18:51:38.282495 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:38.282461    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-97psv_3c108d6c-e58c-4ce8-9d24-9033bb48d08c/download-server/0.log"
Aug 04 18:51:38.808231 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:38.808202    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-zk68b_62793eed-18b9-4c4f-82d9-d75ac2559949/volume-data-source-validator/0.log"
Aug 04 18:51:39.784654 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.784623    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5vwvz_4a651095-5402-42ec-aaae-6bbbfe3f7cf3/dns/0.log"
Aug 04 18:51:39.810848 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.810827    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5vwvz_4a651095-5402-42ec-aaae-6bbbfe3f7cf3/kube-rbac-proxy/0.log"
Aug 04 18:51:39.963166 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963131    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"]
Aug 04 18:51:39.963541 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963493    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963544    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963554    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963560    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963567    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963573    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963583    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963588    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963598    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container"
Aug 04 18:51:39.963600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963611    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963616    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963662    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963672    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87730bd-b669-4029-a664-907e762cdba6" containerName="kserve-container"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963680    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kserve-container"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963686    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kserve-container"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963692    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de5a5a5-de93-459b-8b6c-89d493d3e1f9" containerName="kube-rbac-proxy"
Aug 04 18:51:39.963863 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.963699    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="54eba614-562f-43dd-99f8-ccf29473617b" containerName="kube-rbac-proxy"
Aug 04 18:51:39.966614 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.966598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:39.973358 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.973333    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bh2h8\"/\"kube-root-ca.crt\""
Aug 04 18:51:39.974422 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.974401    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bh2h8\"/\"default-dockercfg-7nj45\""
Aug 04 18:51:39.974539 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:39.974435    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bh2h8\"/\"openshift-service-ca.crt\""
Aug 04 18:51:40.004995 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.004972    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"]
Aug 04 18:51:40.040911 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.040860    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vt5k6_b4681ca7-7610-4a70-951f-850e67bb9c1d/dns-node-resolver/0.log"
Aug 04 18:51:40.043095 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.043078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-sys\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.043158 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.043104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxpxn\" (UniqueName: \"kubernetes.io/projected/2513e47a-db1c-436b-948d-9c0f6a126310-kube-api-access-jxpxn\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.043158 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.043140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-podres\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.043233 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.043161    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-proc\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.043233 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.043202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-lib-modules\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144231 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-lib-modules\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-sys\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxpxn\" (UniqueName: \"kubernetes.io/projected/2513e47a-db1c-436b-948d-9c0f6a126310-kube-api-access-jxpxn\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144319 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-podres\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144434 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144320    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-proc\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144434 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-sys\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144434 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-lib-modules\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144434 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144384    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-proc\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.144615 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.144443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2513e47a-db1c-436b-948d-9c0f6a126310-podres\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.152985 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.152966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxpxn\" (UniqueName: \"kubernetes.io/projected/2513e47a-db1c-436b-948d-9c0f6a126310-kube-api-access-jxpxn\") pod \"perf-node-gather-daemonset-wkd2j\" (UID: \"2513e47a-db1c-436b-948d-9c0f6a126310\") " pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.276901 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.276871    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:40.402321 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.402298    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"]
Aug 04 18:51:40.404633 ip-10-0-136-239 kubenswrapper[2562]: W0804 18:51:40.404603    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2513e47a_db1c_436b_948d_9c0f6a126310.slice/crio-152bcd10a5d9988cf15f8d6e0aa2530b4d7d1c502140d35bd419e54bb8d14519 WatchSource:0}: Error finding container 152bcd10a5d9988cf15f8d6e0aa2530b4d7d1c502140d35bd419e54bb8d14519: Status 404 returned error can't find the container with id 152bcd10a5d9988cf15f8d6e0aa2530b4d7d1c502140d35bd419e54bb8d14519
Aug 04 18:51:40.495253 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.495220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j" event={"ID":"2513e47a-db1c-436b-948d-9c0f6a126310","Type":"ContainerStarted","Data":"152bcd10a5d9988cf15f8d6e0aa2530b4d7d1c502140d35bd419e54bb8d14519"}
Aug 04 18:51:40.575648 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.575593    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-55c9667b59-4dzrf_689ac1de-e938-450c-b0d9-8180e970d9ad/registry/0.log"
Aug 04 18:51:40.656428 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:40.656408    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kkvwm_5204927c-d0ff-4e9b-830e-f06d22586a17/node-ca/0.log"
Aug 04 18:51:41.499574 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:41.499537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j" event={"ID":"2513e47a-db1c-436b-948d-9c0f6a126310","Type":"ContainerStarted","Data":"7a3ddd9201df28518d41dbb57b68cec339071bd60acfa67da24f83ae5e4e89ba"}
Aug 04 18:51:41.499969 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:41.499642    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:41.520852 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:41.520803    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j" podStartSLOduration=2.520788231 podStartE2EDuration="2.520788231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:51:41.518843917 +0000 UTC m=+3061.201292234" watchObservedRunningTime="2026-08-04 18:51:41.520788231 +0000 UTC m=+3061.203236548"
Aug 04 18:51:41.879409 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:41.879368    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v8j5g_52b51669-cef4-45e0-bc42-a96cc97f27e8/serve-healthcheck-canary/0.log"
Aug 04 18:51:42.388795 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:42.388769    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jlxm4_664dc729-b7a8-4f4d-9fd6-fbc54f752945/kube-rbac-proxy/0.log"
Aug 04 18:51:42.413316 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:42.413286    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jlxm4_664dc729-b7a8-4f4d-9fd6-fbc54f752945/exporter/0.log"
Aug 04 18:51:42.441953 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:42.441930    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jlxm4_664dc729-b7a8-4f4d-9fd6-fbc54f752945/extractor/0.log"
Aug 04 18:51:44.875709 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:44.875681    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-mt52k_f5294598-bb18-4c8f-ae88-bd4313a7311d/manager/0.log"
Aug 04 18:51:45.177369 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:45.177289    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-drnvg_809b24a5-9bd3-40a9-9320-b14f4bfa40a6/seaweedfs/0.log"
Aug 04 18:51:47.513986 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:47.513956    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bh2h8/perf-node-gather-daemonset-wkd2j"
Aug 04 18:51:49.749265 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:49.749220    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-k9fd9_5ae694f1-2a76-42fe-9c00-deb6cf23e9b6/migrator/0.log"
Aug 04 18:51:49.776626 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:49.776595    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-k9fd9_5ae694f1-2a76-42fe-9c00-deb6cf23e9b6/graceful-termination/0.log"
Aug 04 18:51:50.175914 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:50.175884    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-xjm4d_386c9d7a-67a4-432d-b226-97305d53924a/kube-storage-version-migrator-operator/1.log"
Aug 04 18:51:50.177551 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:50.177495    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-xjm4d_386c9d7a-67a4-432d-b226-97305d53924a/kube-storage-version-migrator-operator/0.log"
Aug 04 18:51:51.682451 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.682423    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw958_841bbbda-ca31-4cdf-8374-929647c20747/kube-multus-additional-cni-plugins/0.log"
Aug 04 18:51:51.707747 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.707716    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw958_841bbbda-ca31-4cdf-8374-929647c20747/egress-router-binary-copy/0.log"
Aug 04 18:51:51.736577 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.736522    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw958_841bbbda-ca31-4cdf-8374-929647c20747/cni-plugins/0.log"
Aug 04 18:51:51.761581 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.761561    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw958_841bbbda-ca31-4cdf-8374-929647c20747/bond-cni-plugin/0.log"
Aug 04 18:51:51.787266 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.787248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw958_841bbbda-ca31-4cdf-8374-929647c20747/routeoverride-cni/0.log"
Aug 04 18:51:51.811600 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.811586    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw958_841bbbda-ca31-4cdf-8374-929647c20747/whereabouts-cni-bincopy/0.log"
Aug 04 18:51:51.837809 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.837790    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw958_841bbbda-ca31-4cdf-8374-929647c20747/whereabouts-cni/0.log"
Aug 04 18:51:51.926492 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.926473    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x9p9n_d854e521-0559-4697-9e68-a670969d3df1/kube-multus/0.log"
Aug 04 18:51:51.956054 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.956032    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gw46q_4ce8f519-594e-43eb-9e6e-45143be8e3c8/network-metrics-daemon/0.log"
Aug 04 18:51:51.979827 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:51.979804    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gw46q_4ce8f519-594e-43eb-9e6e-45143be8e3c8/kube-rbac-proxy/0.log"
Aug 04 18:51:53.419864 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.419832    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/ovn-controller/0.log"
Aug 04 18:51:53.439582 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.439558    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/ovn-acl-logging/0.log"
Aug 04 18:51:53.468967 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.468944    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/kube-rbac-proxy-node/0.log"
Aug 04 18:51:53.493695 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.493675    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/kube-rbac-proxy-ovn-metrics/0.log"
Aug 04 18:51:53.518557 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.518533    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/northd/0.log"
Aug 04 18:51:53.544664 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.544646    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/nbdb/0.log"
Aug 04 18:51:53.569377 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.569354    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/sbdb/0.log"
Aug 04 18:51:53.626420 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:53.626396    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fnkpr_4f1edb22-437e-44e2-96b1-f7de7c538fe6/ovnkube-controller/0.log"
Aug 04 18:51:54.691987 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:54.691956    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-65bdbbff45-mrq5b_dcfa6f11-3782-4295-bfa7-b3bfa4c61403/check-endpoints/0.log"
Aug 04 18:51:54.749008 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:54.748976    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rvmpt_d5691a2d-bea0-40f6-a2d2-6415a59a4c93/network-check-target-container/0.log"
Aug 04 18:51:55.897315 ip-10-0-136-239 kubenswrapper[2562]: I0804 18:51:55.897286    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-mczn8_e6705ffb-493b-401b-92f7-2ead628737a3/iptables-alerter/0.log"