Sep 18 20:20:18.844071 ip-10-0-137-48 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 20:20:19.295575 ip-10-0-137-48 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:20:19.295575 ip-10-0-137-48 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 20:20:19.295575 ip-10-0-137-48 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:20:19.295575 ip-10-0-137-48 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 20:20:19.295575 ip-10-0-137-48 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:20:19.296477 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.296396    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 20:20:19.299654 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299638    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299656    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299662    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299666    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299670    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299674    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299678    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299681    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299684    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299688    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299691    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299694    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:19.299691 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299697    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299711    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299714    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299717    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299720    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299723    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299726    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299728    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299731    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299734    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299737    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299740    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299742    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299745    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299748    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299751    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299753    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299756    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299759    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299761    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:19.300000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299764    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299767    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299770    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299773    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299775    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299779    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299782    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299784    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299787    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299790    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299792    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299795    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299797    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299800    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299803    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299821    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299826    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299829    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299833    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:19.300560 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299839    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299842    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299845    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299847    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299850    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299852    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299855    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299858    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299861    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299863    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299866    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299869    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299871    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299874    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299877    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299880    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299883    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299886    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299889    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299891    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:19.301088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299894    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299897    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299899    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299902    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299904    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299907    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299910    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299912    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299915    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299918    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299920    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299925    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299928    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299932    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299935    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299937    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299940    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299943    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299946    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299949    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:19.301576 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299952    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299955    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299957    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299960    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299962    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299965    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299968    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.299971    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300647    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300653    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300657    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300660    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300664    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300667    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300669    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300672    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300675    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300678    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300681    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:19.302088 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300684    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300687    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300690    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300692    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300695    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300698    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300701    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300703    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300706    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300709    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300711    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300715    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300719    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300722    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300724    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300727    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300730    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300732    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300735    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300739    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:19.302544 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300742    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300745    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300748    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300751    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300753    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300756    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300759    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300762    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300764    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300767    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300770    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300772    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300776    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300778    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300781    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300784    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300786    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300789    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300792    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300794    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:19.303061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300797    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300800    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300802    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300820    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300823    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300828    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300832    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300835    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300838    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300840    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300843    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300846    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300849    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300852    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300854    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300857    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300860    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300863    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300866    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:19.303593 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300868    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300871    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300874    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300877    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300879    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300882    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300885    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300888    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300891    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300893    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300897    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300900    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300902    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300905    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300908    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300912    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300916    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300919    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300922    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300939    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:19.304071 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300943    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300946    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300949    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300953    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300956    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300959    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300962    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300965    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.300968    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301336    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301342    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301345    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301348    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301351    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301354    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301357    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301360    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301362    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301365    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:19.304566 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301367    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301370    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301374    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301377    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301380    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301383    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301385    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301388    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301391    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301393    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301396    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301399    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301401    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301404    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301406    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301409    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301412    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301415    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301417    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:19.305041 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301420    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301423    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301425    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301428    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301431    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301434    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301436    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301439    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301441    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301444    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301446    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301449    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301452    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301455    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301458    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301460    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301463    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301465    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301468    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301470    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:19.305507 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301473    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301476    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301478    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301481    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301483    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301487    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301489    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301492    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301494    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301497    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301499    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301502    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301505    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301507    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301510    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301512    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301515    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301518    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301521    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301524    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:19.306019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301526    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301529    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301532    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301534    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301537    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301540    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301543    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301546    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301548    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301550    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301553    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301556    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301558    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301561    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301563    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301566    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301568    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301571    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301573    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:19.306523 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301577    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301581    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301583    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301586    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301589    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301592    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301595    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301597    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301600    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301603    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.301606    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301675    2565 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301709    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301715    2565 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301720    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301724    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301727    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301731    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301736    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301739    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301742    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 20:20:19.307007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301746    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301749    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301753    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301756    2565 flags.go:64] FLAG: --cgroup-root=""
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301759    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301762    2565 flags.go:64] FLAG: --client-ca-file=""
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301765    2565 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301768    2565 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301772    2565 flags.go:64] FLAG: --cluster-domain=""
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301774    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301778    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301781    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301784    2565 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301788    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301791    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301794    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301797    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301800    2565 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301803    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301825    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301830    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301833    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301837    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301841    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301844    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 20:20:19.307517 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301847    2565 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301850    2565 flags.go:64] FLAG: --enable-server="true"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301853    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301858    2565 flags.go:64] FLAG: --event-burst="100"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301861    2565 flags.go:64] FLAG: --event-qps="50"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301864    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301867    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301870    2565 flags.go:64] FLAG: --eviction-hard=""
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301874    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301877    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301880    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301883    2565 flags.go:64] FLAG: --eviction-soft=""
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301886    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301889    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301894    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301897    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301900    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301903    2565 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301907    2565 flags.go:64] FLAG: --feature-gates=""
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301910    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301913    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301917    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301920    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301923    2565 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301926    2565 flags.go:64] FLAG: --help="false"
Sep 18 20:20:19.308135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301929    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301933    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301936    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301939    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301943    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301946    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301950    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301953    2565 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301956    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301959    2565 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301962    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301966    2565 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301968    2565 flags.go:64] FLAG: --kube-reserved=""
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301971    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301974    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301977    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301980    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301983    2565 flags.go:64] FLAG: --lock-file=""
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301986    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301989    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301992    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.301999    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302002    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302005    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 20:20:19.308799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302008    2565 flags.go:64] FLAG: --logging-format="text"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302011    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302015    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302018    2565 flags.go:64] FLAG: --manifest-url=""
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302020    2565 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302025    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302028    2565 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302032    2565 flags.go:64] FLAG: --max-pods="110"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302035    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302038    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302041    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302044    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302047    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302050    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302053    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302060    2565 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302063    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302066    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302070    2565 flags.go:64] FLAG: --pod-cidr=""
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302073    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302078    2565 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302081    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302084    2565 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302087    2565 flags.go:64] FLAG: --port="10250"
Sep 18 20:20:19.309435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302090    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302094    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a2c310cb451be89e"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302097    2565 flags.go:64] FLAG: --qos-reserved=""
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302100    2565 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302103    2565 flags.go:64] FLAG: --register-node="true"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302108    2565 flags.go:64] FLAG: --register-with-taints=""
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302112    2565 flags.go:64] FLAG: --registry-burst="10"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302115    2565 flags.go:64] FLAG: --registry-qps="5"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302118    2565 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302120    2565 flags.go:64] FLAG: --reserved-memory=""
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302124    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302127    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302130    2565 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302133    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302136    2565 flags.go:64] FLAG: --runonce="false"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302139    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302142    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302145    2565 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302149    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302152    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302155    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302159    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302162    2565 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302164    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302168    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 20:20:19.310064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302171    2565 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302174    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302177    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302180    2565 flags.go:64] FLAG: --system-cgroups=""
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302183    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302188    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302191    2565 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302194    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302198    2565 flags.go:64] FLAG: --tls-min-version=""
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302201    2565 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302204    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302206    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302211    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302214    2565 flags.go:64] FLAG: --v="2"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302218    2565 flags.go:64] FLAG: --version="false"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302222    2565 flags.go:64] FLAG: --vmodule=""
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302226    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.302229    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302359    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302364    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302368    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302372    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302376    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:19.310726 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302380    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302384    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302389    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302392    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302395    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302398    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302401    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302404    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302407    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302410    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302413    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302416    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302418    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302421    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302424    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302427    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302430    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302432    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302435    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302438    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:19.311606 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302440    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302444    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302447    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302449    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302452    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302455    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302457    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302460    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302463    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302466    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302468    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302471    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302474    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302476    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302480    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302483    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302486    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302488    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302491    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302494    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:19.312305 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302496    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302499    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302502    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302505    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302508    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302511    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302513    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302516    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302518    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302521    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302525    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302528    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302531    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302536    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302539    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302542    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302544    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302547    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302550    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:19.312805 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302553    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302556    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302558    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302561    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302564    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302567    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302570    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302574    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302576    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302579    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302582    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302584    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302587    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302590    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302592    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302595    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302598    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302601    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302604    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302606    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:19.313385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302609    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302611    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302614    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302617    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302619    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302622    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302626    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302629    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302632    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302635    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302638    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302641    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302643    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302646    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.302648    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:19.313901 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.303355    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.311978    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.311994    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.312014    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.312024    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312082    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312087    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312091    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312094    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312097    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312100    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312103    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312106    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312109    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312112    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312114    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312117    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312120    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312122    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:19.314273 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312125    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312128    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312132    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312134    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312137    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312139    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312142    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312145    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312147    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312150    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312153    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312156    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312158    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312161    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312166    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312170    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312173    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312176    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312179    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:19.314743 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312182    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312184    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312187    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312190    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312193    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312196    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312198    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312201    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312204    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312207    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312211    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312214    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312217    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312220    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312223    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312226    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312229    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312232    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312234    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312237    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:19.315228 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312240    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312242    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312246    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312248    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312251    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312254    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312256    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312259    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312262    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312265    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312268    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312270    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312273    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312275    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312278    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312281    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312283    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312286    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312289    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312292    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:19.315767 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312294    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312297    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312301    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312305    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312308    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312311    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312315    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312318    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312321    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312324    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312326    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312329    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312332    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312335    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312338    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312340    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312344    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312347    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312349    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312353    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:19.316279 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312355    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312358    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312361    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312364    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312367    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312369    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.312374    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312490    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312495    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312498    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312501    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312504    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312507    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312510    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:20:19.316944 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312513    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312516    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312518    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312521    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312524    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312528    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312532    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312536    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312539    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312542    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312545    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312548    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312551    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312553    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312556    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312559    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312561    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312565    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312567    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:20:19.317304 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312571    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312573    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312576    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312579    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312581    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312584    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312587    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312590    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312592    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312595    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312598    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312601    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312603    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312606    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312609    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312612    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312614    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312617    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312620    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312623    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:20:19.317761 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312626    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312629    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312632    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312634    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312637    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312640    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312643    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312645    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312648    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312651    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312654    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312657    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312659    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312662    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312665    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312667    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312670    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312672    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312675    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312678    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:20:19.318253 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312681    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312683    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312686    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312688    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312691    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312693    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312696    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312698    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312701    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312704    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312706    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312709    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312713    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312716    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312718    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312721    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312723    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312726    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312729    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312731    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:20:19.318756 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312734    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312737    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312739    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312742    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312744    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312747    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312750    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312753    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312756    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312759    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312761    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312764    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:19.312766    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.312771    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:20:19.319256 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.312902    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 20:20:19.319619 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.315301    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 20:20:19.319619 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.316370    2565 server.go:1019] "Starting client certificate rotation"
Sep 18 20:20:19.319619 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.316468    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:20:19.319619 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.316519    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:20:19.319619 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.319342    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:20:19.320516 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.320498    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:20:19.331148 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.331132    2565 log.go:25] "Validated CRI v1 runtime API"
Sep 18 20:20:19.332927 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.332915    2565 log.go:25] "Validated CRI v1 image API"
Sep 18 20:20:19.333720 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.333704    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 20:20:19.340353 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.340335    2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9a452382-0de7-49d5-adc2-ae885a8c4d4d:/dev/nvme0n1p4 fb3ff89a-fe46-4384-8be1-8523875ac829:/dev/nvme0n1p3]
Sep 18 20:20:19.340435 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.340351    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 20:20:19.346990 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.346869    2565 manager.go:216] Machine: {Timestamp:2026-09-18 20:20:19.345468702 +0000 UTC m=+0.371797841 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103479 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec263f45cbd2f74b1f526a5b71e6f6db SystemUUID:ec263f45-cbd2-f74b-1f52-6a5b71e6f6db BootID:1ed7a8b1-c3dc-46f1-8de6-16ee99d9247e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d5:b4:c5:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d5:b4:c5:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:b5:8f:8e:f1:14 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 20:20:19.346990 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.346982    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 20:20:19.347098 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.347056    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 20:20:19.347597 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.347582    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:20:19.348149 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.348118    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 20:20:19.348287 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.348153    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-48.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 20:20:19.349032 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.349022    2565 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 20:20:19.349063 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.349041    2565 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 20:20:19.349063 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.349049    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:20:19.349680 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.349667    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:20:19.349750 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.349742    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 20:20:19.354395 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.354384    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:20:19.354510 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.354500    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 20:20:19.354553 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.354528    2565 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 20:20:19.354553 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.354540    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 20:20:19.354631 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.354555    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 20:20:19.354631 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.354564    2565 kubelet.go:398] "Adding apiserver pod source"
Sep 18 20:20:19.354631 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.354579    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 20:20:19.355129 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.355100    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:20:19.355129 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.355118    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:20:19.358630 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.358610    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 20:20:19.361539 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.361525    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 20:20:19.361600 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.361555    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 20:20:19.363082 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363070    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363085    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363091    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363097    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363105    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363110    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363123    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363128    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363135    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363140    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363148    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 20:20:19.363158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.363153    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 20:20:19.364621 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.364609    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 20:20:19.364621 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.364622    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 20:20:19.365280 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.365255    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 20:20:19.365713 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.365690    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-48.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 20:20:19.368277 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.368255    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-48.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 20:20:19.368431 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.368418    2565 server.go:1268] "Started kubelet"
Sep 18 20:20:19.368487 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.368465    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 20:20:19.369185 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.369157    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 20:20:19.369287 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.369205    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 20:20:19.369218 ip-10-0-137-48 systemd[1]: Started Kubernetes Kubelet.
Sep 18 20:20:19.371238 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.370762    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 20:20:19.371284 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.371256    2565 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 20:20:19.373306 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.373281    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 20:20:19.379558 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.377580    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-48.ec2.internal.18d683cd38a78949  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-48.ec2.internal,UID:ip-10-0-137-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-48.ec2.internal,},FirstTimestamp:2026-09-18 20:20:19.368388937 +0000 UTC m=+0.394718056,LastTimestamp:2026-09-18 20:20:19.368388937 +0000 UTC m=+0.394718056,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-48.ec2.internal,}"
Sep 18 20:20:19.379909 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.379781    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 20:20:19.379909 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.379797    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 20:20:19.380161 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.380137    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:19.380742 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.380722    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 20:20:19.380885 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.380860    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 20:20:19.380885 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.380878    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 20:20:19.381039 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.380962    2565 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 20:20:19.381039 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.380971    2565 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 20:20:19.381039 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381032    2565 factory.go:55] Registering systemd factory
Sep 18 20:20:19.381177 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381048    2565 factory.go:223] Registration of the systemd container factory successfully
Sep 18 20:20:19.381263 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381251    2565 factory.go:153] Registering CRI-O factory
Sep 18 20:20:19.381294 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381266    2565 factory.go:223] Registration of the crio container factory successfully
Sep 18 20:20:19.381322 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381306    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 20:20:19.381322 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381319    2565 factory.go:103] Registering Raw factory
Sep 18 20:20:19.381377 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381344    2565 manager.go:1196] Started watching for new ooms in manager
Sep 18 20:20:19.381942 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.381928    2565 manager.go:349] Starting recovery of all containers
Sep 18 20:20:19.392275 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.392246    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-48.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 20:20:19.392362 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.392304    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 20:20:19.393238 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.393221    2565 manager.go:354] Recovery completed
Sep 18 20:20:19.397546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.397533    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:19.399299 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.399282    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:19.399375 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.399311    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:19.399375 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.399321    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:19.399878 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.399860    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 20:20:19.399878 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.399876    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 20:20:19.399985 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.399890    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:20:19.399985 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.399867    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:19.400615 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.400602    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:19.400672 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.400626    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:19.400672 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.400636    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:19.402072 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.402055    2565 policy_none.go:49] "None policy: Start"
Sep 18 20:20:19.402072 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.402074    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 20:20:19.402189 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.402087    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 20:20:19.404048 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.404037    2565 policy_none.go:47] "Start"
Sep 18 20:20:19.411203 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.411104    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-48.ec2.internal.18d683cd3a7f2d22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-48.ec2.internal,UID:ip-10-0-137-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-48.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-48.ec2.internal,},FirstTimestamp:2026-09-18 20:20:19.399298338 +0000 UTC m=+0.425627456,LastTimestamp:2026-09-18 20:20:19.399298338 +0000 UTC m=+0.425627456,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-48.ec2.internal,}"
Sep 18 20:20:19.411567 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.411545    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zq72c"
Sep 18 20:20:19.417918 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.417898    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zq72c"
Sep 18 20:20:19.420017 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.419943    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-48.ec2.internal.18d683cd3a7f732d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-48.ec2.internal,UID:ip-10-0-137-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-48.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-48.ec2.internal,},FirstTimestamp:2026-09-18 20:20:19.399316269 +0000 UTC m=+0.425645386,LastTimestamp:2026-09-18 20:20:19.399316269 +0000 UTC m=+0.425645386,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-48.ec2.internal,}"
Sep 18 20:20:19.429347 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.429332    2565 manager.go:337] "Starting Device Plugin manager"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.429364    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.429374    2565 server.go:87] "Starting device plugin registration server"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.429556    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.429567    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.429804    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.429930    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.429949    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.430349    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.430393    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.441721    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.442877    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.442918    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.442940    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.442975    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 20:20:19.454270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.445365    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:19.530549 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.530519    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:19.531879 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.531858    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:19.531991 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.531889    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:19.531991 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.531900    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:19.531991 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.531927    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.542058 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.542042    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.542103 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.542068    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-48.ec2.internal\": node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:19.543194 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.543178    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"]
Sep 18 20:20:19.543249 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.543239    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:19.544780 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.544763    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:19.544876 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.544791    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:19.544876 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.544802    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:19.547152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547096    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:19.547224 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547212    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.547265 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547239    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:19.547873 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547856    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:19.547956 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547877    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:19.547956 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547892    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:19.547956 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547898    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:19.547956 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547907    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:19.547956 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.547908    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:19.550170 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.550153    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.550256 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.550188    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:20:19.550921 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.550904    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:20:19.550999 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.550930    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:20:19.550999 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.550944    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:20:19.571360 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.571335    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:19.581529 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.581513    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-48.ec2.internal\" not found" node="ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.585759 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.585744    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-48.ec2.internal\" not found" node="ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.671942 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.671913    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:19.682221 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.682196    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31e58ae76b01e03c1d1e7d812b452870-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal\" (UID: \"31e58ae76b01e03c1d1e7d812b452870\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.682221 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.682223    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2855f1593c0393be173039d69f8435c0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-48.ec2.internal\" (UID: \"2855f1593c0393be173039d69f8435c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.682383 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.682240    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31e58ae76b01e03c1d1e7d812b452870-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal\" (UID: \"31e58ae76b01e03c1d1e7d812b452870\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.772382 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.772344    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:19.782696 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.782667    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31e58ae76b01e03c1d1e7d812b452870-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal\" (UID: \"31e58ae76b01e03c1d1e7d812b452870\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.782759 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.782700    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31e58ae76b01e03c1d1e7d812b452870-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal\" (UID: \"31e58ae76b01e03c1d1e7d812b452870\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.782759 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.782716    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2855f1593c0393be173039d69f8435c0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-48.ec2.internal\" (UID: \"2855f1593c0393be173039d69f8435c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.782857 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.782759    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31e58ae76b01e03c1d1e7d812b452870-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal\" (UID: \"31e58ae76b01e03c1d1e7d812b452870\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.782857 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.782836    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2855f1593c0393be173039d69f8435c0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-48.ec2.internal\" (UID: \"2855f1593c0393be173039d69f8435c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.782921 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.782863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31e58ae76b01e03c1d1e7d812b452870-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal\" (UID: \"31e58ae76b01e03c1d1e7d812b452870\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.873169 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.873096    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:19.886581 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.886551    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.890821 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:19.890788    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:19.974270 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:19.974108    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:20.074651 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:20.074608    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:21.326605 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:21.326523    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:21.427168 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:21.427133    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-48.ec2.internal\" not found"
Sep 18 20:20:21.467599 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.467578    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 20:20:21.467996 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.467712    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 20:20:21.507458 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.507440    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:21.532351 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.532327    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:21.532351 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.532339    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 20:20:21.545224 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.545183    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:20:21.552062 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.552044    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:20:21.552898 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.552881    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:21.553572 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.553439    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal"
Sep 18 20:20:21.565395 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.565366    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:20:21.575661 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.575616    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 20:15:20 +0000 UTC" deadline="2028-06-15 20:47:00.967799872 +0000 UTC"
Sep 18 20:20:21.575712 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.575661    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15264h26m39.392141347s"
Sep 18 20:20:21.579063 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.579006    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8mbkb"
Sep 18 20:20:21.588689 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.588671    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8mbkb"
Sep 18 20:20:21.645168 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.645141    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:21.653439 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:21.653410    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2855f1593c0393be173039d69f8435c0.slice/crio-1e93742a05614c8ba0e1cc3d5879c95b5285a156689d8f469418aedde635a7ee WatchSource:0}: Error finding container 1e93742a05614c8ba0e1cc3d5879c95b5285a156689d8f469418aedde635a7ee: Status 404 returned error can't find the container with id 1e93742a05614c8ba0e1cc3d5879c95b5285a156689d8f469418aedde635a7ee
Sep 18 20:20:21.653876 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:21.653854    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e58ae76b01e03c1d1e7d812b452870.slice/crio-0d0112cef693ca821180fbe3a78452ffafbd9d18e565a88589520ebdb3c5c38f WatchSource:0}: Error finding container 0d0112cef693ca821180fbe3a78452ffafbd9d18e565a88589520ebdb3c5c38f: Status 404 returned error can't find the container with id 0d0112cef693ca821180fbe3a78452ffafbd9d18e565a88589520ebdb3c5c38f
Sep 18 20:20:21.660018 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:21.659990    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:20:22.506587 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.506549    2565 apiserver.go:52] "Watching apiserver"
Sep 18 20:20:22.514971 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.514948    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 20:20:22.515376 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.515353    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-bkvcr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal","openshift-multus/multus-ljrz9","openshift-network-operator/iptables-alerter-c9zng","kube-system/global-pull-secret-syncer-qp8n2","kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal","openshift-dns/node-resolver-lkq9j","openshift-image-registry/node-ca-8fkmr","openshift-multus/multus-additional-cni-plugins-xznrr","openshift-multus/network-metrics-daemon-l56h5","openshift-network-diagnostics/network-check-target-9rqxn","openshift-ovn-kubernetes/ovnkube-node-vgfrj","kube-system/konnectivity-agent-fgtsn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"]
Sep 18 20:20:22.518304 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.518278    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:22.518394 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.518368    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:22.520457 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.520437    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.523301 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.522457    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 20:20:22.523301 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.522582    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 20:20:22.523301 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.522784    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sfhnx\""
Sep 18 20:20:22.523490 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.523445    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.525260 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.525235    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 20:20:22.526678 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.525681    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l49cp\""
Sep 18 20:20:22.526678 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.525860    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 20:20:22.526678 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.526016    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 20:20:22.526678 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.526167    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 20:20:22.527527 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.527058    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.529198 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.529026    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n98ss\""
Sep 18 20:20:22.529296 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.529279    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 20:20:22.529353 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.529293    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 20:20:22.529353 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.529306    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 20:20:22.530432 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.530412    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.532443 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.532281    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 20:20:22.532443 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.532397    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 20:20:22.532443 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.532435    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mmwjw\""
Sep 18 20:20:22.532674 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.532504    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 20:20:22.532829 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.532808    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.534636 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.534615    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 20:20:22.534712 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.534644    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 20:20:22.534758 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.534710    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 20:20:22.534816 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.534788    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 20:20:22.535047 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.535029    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 20:20:22.535119 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.535033    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vcv59\""
Sep 18 20:20:22.535119 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.535095    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 20:20:22.535411 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.535392    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.538142 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.538117    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.540011 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.539550    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dskct\""
Sep 18 20:20:22.540011 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.539664    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 20:20:22.540011 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.539909    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 20:20:22.541476 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.541121    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sm4r4\""
Sep 18 20:20:22.541476 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.541382    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 20:20:22.541630 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.541468    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 20:20:22.543259 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543225    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-ovn\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.543348 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543264    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.543348 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543293    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q95gt\" (UniqueName: \"kubernetes.io/projected/1c08e560-dab1-4bf9-b1a4-beec6c664021-kube-api-access-q95gt\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:22.543348 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-k8s-cni-cncf-io\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543348 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543340    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-cni-bin\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-device-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.543546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543380    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-var-lib-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.543546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543400    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-system-cni-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543421    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-cni-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543458    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-cni-multus\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543503    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-multus-certs\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543549    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-tmp-dir\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543558    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543586    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-run-netns\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543611    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-env-overrides\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543635    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-cnibin\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543675    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-socket-dir-parent\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543714    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-hostroot\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543743    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543769    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-kubelet\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-systemd-units\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.543825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543812    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543848    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543880    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovn-node-metrics-cert\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543911    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-netns\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543947    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.543987    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-registration-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544038    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-systemd\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544079    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-etc-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544103    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-sys-fs\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544140    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjm7l\" (UniqueName: \"kubernetes.io/projected/382faecf-06e9-449f-b0f8-7028aaabe9b5-kube-api-access-rjm7l\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544164    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f23a7662-58e8-41a3-80e3-bcf87e30af10-host\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-node-log\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544245    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-log-socket\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544279    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvl86\" (UniqueName: \"kubernetes.io/projected/85eb79b8-f47b-40da-a41c-94b053ddc4a3-kube-api-access-zvl86\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544306    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544329    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-kubelet\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.544354 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544350    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/803a9562-345e-4a66-9a13-095c205d61ee-multus-daemon-config\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544372    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2m5s\" (UniqueName: \"kubernetes.io/projected/803a9562-345e-4a66-9a13-095c205d61ee-kube-api-access-n2m5s\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544392    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f23a7662-58e8-41a3-80e3-bcf87e30af10-serviceca\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544416    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bftdw\" (UniqueName: \"kubernetes.io/projected/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-kube-api-access-bftdw\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544440    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-slash\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544490    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-cni-bin\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-cni-netd\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovnkube-config\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544559    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovnkube-script-lib\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27wsp\" (UniqueName: \"kubernetes.io/projected/f23a7662-58e8-41a3-80e3-bcf87e30af10-kube-api-access-27wsp\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544589    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-hosts-file\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544622    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/803a9562-345e-4a66-9a13-095c205d61ee-cni-binary-copy\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544651    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-conf-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544748    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-etc-kubernetes\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.545166 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.544776    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-socket-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.545833 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.545361    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tb9f9\""
Sep 18 20:20:22.545833 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.545397    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:20:22.545833 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.545622    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 20:20:22.546046 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.546026    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:22.546108 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.546088    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:22.548535 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.548517    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.548704 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.548681    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:22.551070 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.551048    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.553031 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.553011    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 20:20:22.553460 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.553414    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cph4m\""
Sep 18 20:20:22.553576 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.553547    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:20:22.553707 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.553692    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 20:20:22.592487 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.592457    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 20:15:21 +0000 UTC" deadline="2028-05-07 03:08:30.211109182 +0000 UTC"
Sep 18 20:20:22.592487 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.592487    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14310h48m7.618626659s"
Sep 18 20:20:22.598980 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.598939    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal" event={"ID":"31e58ae76b01e03c1d1e7d812b452870","Type":"ContainerStarted","Data":"0d0112cef693ca821180fbe3a78452ffafbd9d18e565a88589520ebdb3c5c38f"}
Sep 18 20:20:22.600070 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.600052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal" event={"ID":"2855f1593c0393be173039d69f8435c0","Type":"ContainerStarted","Data":"1e93742a05614c8ba0e1cc3d5879c95b5285a156689d8f469418aedde635a7ee"}
Sep 18 20:20:22.605123 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.605101    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:20:22.633782 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.633754    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 20:20:22.645860 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.645834    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysctl-d\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.645983 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.645868    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-system-cni-dir\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.645983 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.645906    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-systemd-units\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.645983 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.645930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.645983 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.645974    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-systemd-units\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646159 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646159 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646063    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646159 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646159 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646128    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-registration-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.646159 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646154    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjm7l\" (UniqueName: \"kubernetes.io/projected/382faecf-06e9-449f-b0f8-7028aaabe9b5-kube-api-access-rjm7l\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-host\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646191    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-systemd\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646226    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-registration-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-sys-fs\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646279    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-sys-fs\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysconfig\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646329    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-systemd\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646353    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-kubelet\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646387    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f23a7662-58e8-41a3-80e3-bcf87e30af10-serviceca\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-slash\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646451 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646436    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-cni-bin\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646476    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-kubelet\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-conf-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-slash\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-etc-kubernetes\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646538    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-cni-bin\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646563    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-socket-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646589    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-conf-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646593    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646635    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-etc-kubernetes\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-ovn\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646737    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-k8s-cni-cncf-io\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646752    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-socket-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646763    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-cni-bin\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.646948 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646801    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-run-ovn\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646793    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646855    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-k8s-cni-cncf-io\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-var-lib-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-cni-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646938    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-cni-multus\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646958    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-var-lib-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646984    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.646971    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-cni-multus\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647018    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-multus-certs\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647034    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-var-lib-cni-bin\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647092    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-sys\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-multus-certs\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647176    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-cni-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647388    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f23a7662-58e8-41a3-80e3-bcf87e30af10-serviceca\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647167    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0e45d494-de53-40cb-9602-aa50960e4ca8-iptables-alerter-script\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.647743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647695    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-run-netns\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647714    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-env-overrides\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647735    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-cnibin\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-socket-dir-parent\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647792    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-kubelet\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647801    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-run-netns\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovn-node-metrics-cert\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-netns\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647868    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-cnibin\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-multus-socket-dir-parent\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647897    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-kubelet\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647957    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-host-run-netns\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647973    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.647982    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f23a7662-58e8-41a3-80e3-bcf87e30af10-host\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648011    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-modprobe-d\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648018    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f23a7662-58e8-41a3-80e3-bcf87e30af10-host\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.648509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648035    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-var-lib-kubelet\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648059    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-cnibin\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648073    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648085    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-etc-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648120    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysctl-conf\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648135    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-etc-openvswitch\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648144    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3f61d6b-591b-47fb-9087-c422a223031f-etc-tuned\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e45d494-de53-40cb-9602-aa50960e4ca8-host-slash\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648253    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648266    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-env-overrides\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-node-log\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648304    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-node-log\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648314    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-log-socket\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648337    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvl86\" (UniqueName: \"kubernetes.io/projected/85eb79b8-f47b-40da-a41c-94b053ddc4a3-kube-api-access-zvl86\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648345    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-log-socket\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:22.649343 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648377    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/803a9562-345e-4a66-9a13-095c205d61ee-multus-daemon-config\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648396    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2m5s\" (UniqueName: \"kubernetes.io/projected/803a9562-345e-4a66-9a13-095c205d61ee-kube-api-access-n2m5s\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648416    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-run\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.648839    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-kubelet-config\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bftdw\" (UniqueName: \"kubernetes.io/projected/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-kube-api-access-bftdw\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649080    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-cni-netd\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.649238    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.649326    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs podName:1c08e560-dab1-4bf9-b1a4-beec6c664021 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:23.149291511 +0000 UTC m=+3.024221576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs") pod "network-metrics-daemon-l56h5" (UID: "1c08e560-dab1-4bf9-b1a4-beec6c664021") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649365    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovnkube-config\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649393    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovnkube-script-lib\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27wsp\" (UniqueName: \"kubernetes.io/projected/f23a7662-58e8-41a3-80e3-bcf87e30af10-kube-api-access-27wsp\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649460    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a723e0f7-c700-46c0-ae2a-73ce8f59dff9-konnectivity-ca\") pod \"konnectivity-agent-fgtsn\" (UID: \"a723e0f7-c700-46c0-ae2a-73ce8f59dff9\") " pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649462    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/803a9562-345e-4a66-9a13-095c205d61ee-multus-daemon-config\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649490    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-dbus\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-hosts-file\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.650066 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.649549    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/803a9562-345e-4a66-9a13-095c205d61ee-cni-binary-copy\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.650724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650282    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/803a9562-345e-4a66-9a13-095c205d61ee-cni-binary-copy\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.650724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650282    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-hosts-file\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.650724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-lib-modules\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.650724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovnkube-script-lib\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.650724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650583    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85eb79b8-f47b-40da-a41c-94b053ddc4a3-host-cni-netd\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.650724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f61d6b-591b-47fb-9087-c422a223031f-tmp\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.650724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnmjt\" (UniqueName: \"kubernetes.io/projected/a3f61d6b-591b-47fb-9087-c422a223031f-kube-api-access-rnmjt\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95gt\" (UniqueName: \"kubernetes.io/projected/1c08e560-dab1-4bf9-b1a4-beec6c664021-kube-api-access-q95gt\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650798    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-device-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650802    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovnkube-config\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650879    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/382faecf-06e9-449f-b0f8-7028aaabe9b5-device-dir\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-systemd\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrbp5\" (UniqueName: \"kubernetes.io/projected/0e45d494-de53-40cb-9602-aa50960e4ca8-kube-api-access-jrbp5\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.650977    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkhgz\" (UniqueName: \"kubernetes.io/projected/76292b1c-a487-4039-8548-1cadba4c8fe4-kube-api-access-gkhgz\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.651013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651012    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-system-cni-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.651418 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-tmp-dir\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.651418 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651099    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-hostroot\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.651418 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651130    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a723e0f7-c700-46c0-ae2a-73ce8f59dff9-agent-certs\") pod \"konnectivity-agent-fgtsn\" (UID: \"a723e0f7-c700-46c0-ae2a-73ce8f59dff9\") " pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.651418 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651141    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-system-cni-dir\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.651418 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-kubernetes\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.651418 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651199    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/803a9562-345e-4a66-9a13-095c205d61ee-hostroot\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.651683 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.651475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-tmp-dir\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.654287 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.654266    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85eb79b8-f47b-40da-a41c-94b053ddc4a3-ovn-node-metrics-cert\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.654458 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.654438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjm7l\" (UniqueName: \"kubernetes.io/projected/382faecf-06e9-449f-b0f8-7028aaabe9b5-kube-api-access-rjm7l\") pod \"aws-ebs-csi-driver-node-9td4g\" (UID: \"382faecf-06e9-449f-b0f8-7028aaabe9b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.657477 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.657451    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2m5s\" (UniqueName: \"kubernetes.io/projected/803a9562-345e-4a66-9a13-095c205d61ee-kube-api-access-n2m5s\") pod \"multus-ljrz9\" (UID: \"803a9562-345e-4a66-9a13-095c205d61ee\") " pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.657670 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.657648    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bftdw\" (UniqueName: \"kubernetes.io/projected/6fce8f58-7f3c-402c-bd7f-0318ccf013d3-kube-api-access-bftdw\") pod \"node-resolver-lkq9j\" (UID: \"6fce8f58-7f3c-402c-bd7f-0318ccf013d3\") " pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.657847 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.657825    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27wsp\" (UniqueName: \"kubernetes.io/projected/f23a7662-58e8-41a3-80e3-bcf87e30af10-kube-api-access-27wsp\") pod \"node-ca-8fkmr\" (UID: \"f23a7662-58e8-41a3-80e3-bcf87e30af10\") " pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.658115 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.658094    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvl86\" (UniqueName: \"kubernetes.io/projected/85eb79b8-f47b-40da-a41c-94b053ddc4a3-kube-api-access-zvl86\") pod \"ovnkube-node-vgfrj\" (UID: \"85eb79b8-f47b-40da-a41c-94b053ddc4a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.659612 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.659595    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q95gt\" (UniqueName: \"kubernetes.io/projected/1c08e560-dab1-4bf9-b1a4-beec6c664021-kube-api-access-q95gt\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:22.752197 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752166    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-run\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752197 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752197    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-kubelet-config\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752227    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a723e0f7-c700-46c0-ae2a-73ce8f59dff9-konnectivity-ca\") pod \"konnectivity-agent-fgtsn\" (UID: \"a723e0f7-c700-46c0-ae2a-73ce8f59dff9\") " pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752300    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-kubelet-config\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752299    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-run\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752325    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-dbus\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752359    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-lib-modules\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752372    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-dbus\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752381    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f61d6b-591b-47fb-9087-c422a223031f-tmp\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752405    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnmjt\" (UniqueName: \"kubernetes.io/projected/a3f61d6b-591b-47fb-9087-c422a223031f-kube-api-access-rnmjt\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-systemd\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrbp5\" (UniqueName: \"kubernetes.io/projected/0e45d494-de53-40cb-9602-aa50960e4ca8-kube-api-access-jrbp5\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752476    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkhgz\" (UniqueName: \"kubernetes.io/projected/76292b1c-a487-4039-8548-1cadba4c8fe4-kube-api-access-gkhgz\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752489    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-lib-modules\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a723e0f7-c700-46c0-ae2a-73ce8f59dff9-agent-certs\") pod \"konnectivity-agent-fgtsn\" (UID: \"a723e0f7-c700-46c0-ae2a-73ce8f59dff9\") " pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-kubernetes\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752552    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysctl-d\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752576    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-system-cni-dir\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-host\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysconfig\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752670    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysconfig\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752709    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-host\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752714    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752741    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752781    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-kubernetes\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.752914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-sys\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752800    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a723e0f7-c700-46c0-ae2a-73ce8f59dff9-konnectivity-ca\") pod \"konnectivity-agent-fgtsn\" (UID: \"a723e0f7-c700-46c0-ae2a-73ce8f59dff9\") " pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.752901    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752910    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysctl-d\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.752956    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret podName:0d1d26c8-baa0-4bc3-97bf-ed62049a59f5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:23.252938049 +0000 UTC m=+3.127868103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret") pod "global-pull-secret-syncer-qp8n2" (UID: "0d1d26c8-baa0-4bc3-97bf-ed62049a59f5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-sys\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.752828    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0e45d494-de53-40cb-9602-aa50960e4ca8-iptables-alerter-script\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753035    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-system-cni-dir\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753117    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753159    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-modprobe-d\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-var-lib-kubelet\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-cnibin\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753283    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysctl-conf\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753293    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753320    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-modprobe-d\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753324    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753347    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-var-lib-kubelet\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.753808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753329    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3f61d6b-591b-47fb-9087-c422a223031f-etc-tuned\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76292b1c-a487-4039-8548-1cadba4c8fe4-cnibin\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753396    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e45d494-de53-40cb-9602-aa50960e4ca8-host-slash\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753423    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753460    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-sysctl-conf\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753485    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e45d494-de53-40cb-9602-aa50960e4ca8-host-slash\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753528    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3f61d6b-591b-47fb-9087-c422a223031f-etc-systemd\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753577    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0e45d494-de53-40cb-9602-aa50960e4ca8-iptables-alerter-script\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753639    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.754642 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.753878    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76292b1c-a487-4039-8548-1cadba4c8fe4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.755135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.755112    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f61d6b-591b-47fb-9087-c422a223031f-tmp\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.755486 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.755463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3f61d6b-591b-47fb-9087-c422a223031f-etc-tuned\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.755798 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.755777    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a723e0f7-c700-46c0-ae2a-73ce8f59dff9-agent-certs\") pod \"konnectivity-agent-fgtsn\" (UID: \"a723e0f7-c700-46c0-ae2a-73ce8f59dff9\") " pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.761485 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.761161    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:22.761485 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.761184    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:22.761485 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.761199    2565 projected.go:196] Error preparing data for projected volume kube-api-access-7z7q2 for pod openshift-network-diagnostics/network-check-target-9rqxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:22.761485 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:22.761285    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2 podName:34b1b8ba-0fdc-433a-afab-64580eb8c2b2 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:23.26126858 +0000 UTC m=+3.136198636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7z7q2" (UniqueName: "kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2") pod "network-check-target-9rqxn" (UID: "34b1b8ba-0fdc-433a-afab-64580eb8c2b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:22.762994 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.762969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrbp5\" (UniqueName: \"kubernetes.io/projected/0e45d494-de53-40cb-9602-aa50960e4ca8-kube-api-access-jrbp5\") pod \"iptables-alerter-c9zng\" (UID: \"0e45d494-de53-40cb-9602-aa50960e4ca8\") " pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:22.763311 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.763289    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnmjt\" (UniqueName: \"kubernetes.io/projected/a3f61d6b-591b-47fb-9087-c422a223031f-kube-api-access-rnmjt\") pod \"tuned-bkvcr\" (UID: \"a3f61d6b-591b-47fb-9087-c422a223031f\") " pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.763743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.763726    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkhgz\" (UniqueName: \"kubernetes.io/projected/76292b1c-a487-4039-8548-1cadba4c8fe4-kube-api-access-gkhgz\") pod \"multus-additional-cni-plugins-xznrr\" (UID: \"76292b1c-a487-4039-8548-1cadba4c8fe4\") " pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.832872 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.832836    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkq9j"
Sep 18 20:20:22.843412 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.843391    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljrz9"
Sep 18 20:20:22.854073 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.854054    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:22.859462 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.859441    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g"
Sep 18 20:20:22.864727 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.864706    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8fkmr"
Sep 18 20:20:22.871351 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.871332    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xznrr"
Sep 18 20:20:22.876788 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.876771    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:22.882229 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.882195    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkvcr"
Sep 18 20:20:22.886669 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:22.886653    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9zng"
Sep 18 20:20:23.155584 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.155515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:23.155727 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.155635    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:23.155727 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.155694    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs podName:1c08e560-dab1-4bf9-b1a4-beec6c664021 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:24.155681055 +0000 UTC m=+4.030611096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs") pod "network-metrics-daemon-l56h5" (UID: "1c08e560-dab1-4bf9-b1a4-beec6c664021") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:23.255847 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.255808    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:23.256024 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.255974    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:23.256103 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.256052    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret podName:0d1d26c8-baa0-4bc3-97bf-ed62049a59f5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:24.256034311 +0000 UTC m=+4.130964377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret") pod "global-pull-secret-syncer-qp8n2" (UID: "0d1d26c8-baa0-4bc3-97bf-ed62049a59f5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:23.356872 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.356837    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:23.357044 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.357026    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:23.357121 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.357049    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:23.357121 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.357064    2565 projected.go:196] Error preparing data for projected volume kube-api-access-7z7q2 for pod openshift-network-diagnostics/network-check-target-9rqxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:23.357244 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.357143    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2 podName:34b1b8ba-0fdc-433a-afab-64580eb8c2b2 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:24.357123872 +0000 UTC m=+4.232053932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7z7q2" (UniqueName: "kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2") pod "network-check-target-9rqxn" (UID: "34b1b8ba-0fdc-433a-afab-64580eb8c2b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:23.565194 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.565171    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fce8f58_7f3c_402c_bd7f_0318ccf013d3.slice/crio-be5e5a50295fea3bfb23e8ee5d0a87d76cd519462bc975d6e944343f2f0d6b99 WatchSource:0}: Error finding container be5e5a50295fea3bfb23e8ee5d0a87d76cd519462bc975d6e944343f2f0d6b99: Status 404 returned error can't find the container with id be5e5a50295fea3bfb23e8ee5d0a87d76cd519462bc975d6e944343f2f0d6b99
Sep 18 20:20:23.566615 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.566588    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76292b1c_a487_4039_8548_1cadba4c8fe4.slice/crio-07bb9c799de14e3770964f76cc149dd5794fada289e609ede7ddd1ba2ac8a36d WatchSource:0}: Error finding container 07bb9c799de14e3770964f76cc149dd5794fada289e609ede7ddd1ba2ac8a36d: Status 404 returned error can't find the container with id 07bb9c799de14e3770964f76cc149dd5794fada289e609ede7ddd1ba2ac8a36d
Sep 18 20:20:23.567385 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.567362    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda723e0f7_c700_46c0_ae2a_73ce8f59dff9.slice/crio-4fd8f6731092032ba63a5904ee47bb7853fc2df9f984615ada742fa5c7fffee3 WatchSource:0}: Error finding container 4fd8f6731092032ba63a5904ee47bb7853fc2df9f984615ada742fa5c7fffee3: Status 404 returned error can't find the container with id 4fd8f6731092032ba63a5904ee47bb7853fc2df9f984615ada742fa5c7fffee3
Sep 18 20:20:23.568936 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.568819    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23a7662_58e8_41a3_80e3_bcf87e30af10.slice/crio-be4b970620f27e38fbd4ae4016b86567c905616a929deae3e4c516175063c11f WatchSource:0}: Error finding container be4b970620f27e38fbd4ae4016b86567c905616a929deae3e4c516175063c11f: Status 404 returned error can't find the container with id be4b970620f27e38fbd4ae4016b86567c905616a929deae3e4c516175063c11f
Sep 18 20:20:23.571156 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.571129    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e45d494_de53_40cb_9602_aa50960e4ca8.slice/crio-942a16f3c1b407598525a43dfb3a43044cbbe0b062e703868ca59fa8ced3c6cf WatchSource:0}: Error finding container 942a16f3c1b407598525a43dfb3a43044cbbe0b062e703868ca59fa8ced3c6cf: Status 404 returned error can't find the container with id 942a16f3c1b407598525a43dfb3a43044cbbe0b062e703868ca59fa8ced3c6cf
Sep 18 20:20:23.572591 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.572556    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod382faecf_06e9_449f_b0f8_7028aaabe9b5.slice/crio-c40064566b220f91f1441c414284b858ff3a698ee24e1862a8f302af24f1c4bb WatchSource:0}: Error finding container c40064566b220f91f1441c414284b858ff3a698ee24e1862a8f302af24f1c4bb: Status 404 returned error can't find the container with id c40064566b220f91f1441c414284b858ff3a698ee24e1862a8f302af24f1c4bb
Sep 18 20:20:23.573464 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.573433    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85eb79b8_f47b_40da_a41c_94b053ddc4a3.slice/crio-0f78424eb43b83b2d6efabb522b987c4474df3b7de2f18cb07dd9e4e0331d35f WatchSource:0}: Error finding container 0f78424eb43b83b2d6efabb522b987c4474df3b7de2f18cb07dd9e4e0331d35f: Status 404 returned error can't find the container with id 0f78424eb43b83b2d6efabb522b987c4474df3b7de2f18cb07dd9e4e0331d35f
Sep 18 20:20:23.573988 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.573963    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803a9562_345e_4a66_9a13_095c205d61ee.slice/crio-85b7e77336448fa810a53d0a40d7f7f4b14819a22cf0ca6cc7d7afa07fd3c174 WatchSource:0}: Error finding container 85b7e77336448fa810a53d0a40d7f7f4b14819a22cf0ca6cc7d7afa07fd3c174: Status 404 returned error can't find the container with id 85b7e77336448fa810a53d0a40d7f7f4b14819a22cf0ca6cc7d7afa07fd3c174
Sep 18 20:20:23.575638 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:23.575543    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f61d6b_591b_47fb_9087_c422a223031f.slice/crio-f7f8336283ce9e6102a17742d1ea0d26da87164eabc51b925796fb80b3495b87 WatchSource:0}: Error finding container f7f8336283ce9e6102a17742d1ea0d26da87164eabc51b925796fb80b3495b87: Status 404 returned error can't find the container with id f7f8336283ce9e6102a17742d1ea0d26da87164eabc51b925796fb80b3495b87
Sep 18 20:20:23.593557 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.593513    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 20:15:21 +0000 UTC" deadline="2028-04-24 00:02:13.529925076 +0000 UTC"
Sep 18 20:20:23.593557 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.593555    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13995h41m49.936372601s"
Sep 18 20:20:23.594681 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.594663    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:23.594767 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:23.594750    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:23.604545 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.604519    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljrz9" event={"ID":"803a9562-345e-4a66-9a13-095c205d61ee","Type":"ContainerStarted","Data":"85b7e77336448fa810a53d0a40d7f7f4b14819a22cf0ca6cc7d7afa07fd3c174"}
Sep 18 20:20:23.611438 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.611413    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"0f78424eb43b83b2d6efabb522b987c4474df3b7de2f18cb07dd9e4e0331d35f"}
Sep 18 20:20:23.613904 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.613875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g" event={"ID":"382faecf-06e9-449f-b0f8-7028aaabe9b5","Type":"ContainerStarted","Data":"c40064566b220f91f1441c414284b858ff3a698ee24e1862a8f302af24f1c4bb"}
Sep 18 20:20:23.614975 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.614953    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9zng" event={"ID":"0e45d494-de53-40cb-9602-aa50960e4ca8","Type":"ContainerStarted","Data":"942a16f3c1b407598525a43dfb3a43044cbbe0b062e703868ca59fa8ced3c6cf"}
Sep 18 20:20:23.616075 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.616051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkvcr" event={"ID":"a3f61d6b-591b-47fb-9087-c422a223031f","Type":"ContainerStarted","Data":"f7f8336283ce9e6102a17742d1ea0d26da87164eabc51b925796fb80b3495b87"}
Sep 18 20:20:23.617239 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.617196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8fkmr" event={"ID":"f23a7662-58e8-41a3-80e3-bcf87e30af10","Type":"ContainerStarted","Data":"be4b970620f27e38fbd4ae4016b86567c905616a929deae3e4c516175063c11f"}
Sep 18 20:20:23.618200 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.618183    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fgtsn" event={"ID":"a723e0f7-c700-46c0-ae2a-73ce8f59dff9","Type":"ContainerStarted","Data":"4fd8f6731092032ba63a5904ee47bb7853fc2df9f984615ada742fa5c7fffee3"}
Sep 18 20:20:23.619254 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.619234    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerStarted","Data":"07bb9c799de14e3770964f76cc149dd5794fada289e609ede7ddd1ba2ac8a36d"}
Sep 18 20:20:23.620232 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:23.620200    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkq9j" event={"ID":"6fce8f58-7f3c-402c-bd7f-0318ccf013d3","Type":"ContainerStarted","Data":"be5e5a50295fea3bfb23e8ee5d0a87d76cd519462bc975d6e944343f2f0d6b99"}
Sep 18 20:20:24.162683 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.161796    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:24.162683 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.162195    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:24.162683 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.162276    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs podName:1c08e560-dab1-4bf9-b1a4-beec6c664021 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:26.162256598 +0000 UTC m=+6.037186655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs") pod "network-metrics-daemon-l56h5" (UID: "1c08e560-dab1-4bf9-b1a4-beec6c664021") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:24.263390 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.263347    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:24.263569 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.263538    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:24.263630 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.263601    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret podName:0d1d26c8-baa0-4bc3-97bf-ed62049a59f5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:26.263583606 +0000 UTC m=+6.138513651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret") pod "global-pull-secret-syncer-qp8n2" (UID: "0d1d26c8-baa0-4bc3-97bf-ed62049a59f5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:24.363783 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.363743    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:24.363967 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.363952    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:24.364031 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.363972    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:24.364031 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.363984    2565 projected.go:196] Error preparing data for projected volume kube-api-access-7z7q2 for pod openshift-network-diagnostics/network-check-target-9rqxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:24.364130 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.364054    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2 podName:34b1b8ba-0fdc-433a-afab-64580eb8c2b2 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:26.364023909 +0000 UTC m=+6.238953956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7z7q2" (UniqueName: "kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2") pod "network-check-target-9rqxn" (UID: "34b1b8ba-0fdc-433a-afab-64580eb8c2b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:24.597333 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.595437    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:24.597333 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.595589    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:24.597333 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.595986    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:24.597333 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:24.596077    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:24.625717 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.625684    2565 generic.go:356] "Generic (PLEG): container finished" podID="31e58ae76b01e03c1d1e7d812b452870" containerID="340dd418566f7b53294c544a53c9e9344375c4074754bccc60eaa6136442b2d8" exitCode=0
Sep 18 20:20:24.625851 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.625765    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal" event={"ID":"31e58ae76b01e03c1d1e7d812b452870","Type":"ContainerDied","Data":"340dd418566f7b53294c544a53c9e9344375c4074754bccc60eaa6136442b2d8"}
Sep 18 20:20:24.630162 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.629787    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal" event={"ID":"2855f1593c0393be173039d69f8435c0","Type":"ContainerStarted","Data":"2525e7b4e860ac7ffcee09da33b0d156bb98e98bd5eb790caaa49fc45a0ac515"}
Sep 18 20:20:24.656823 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:24.656764    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-48.ec2.internal" podStartSLOduration=3.656745218 podStartE2EDuration="3.656745218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:24.65641496 +0000 UTC m=+4.531345025" watchObservedRunningTime="2026-09-18 20:20:24.656745218 +0000 UTC m=+4.531675283"
Sep 18 20:20:25.595766 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:25.595280    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:25.595766 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:25.595402    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:25.663809 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:25.663740    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal" event={"ID":"31e58ae76b01e03c1d1e7d812b452870","Type":"ContainerStarted","Data":"d951b83855681537ca94c401b1ed0953272ab33c56a58a2621ec56d3bb87e353"}
Sep 18 20:20:26.178756 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:26.178717    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:26.178947 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.178866    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:26.179007 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.178953    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs podName:1c08e560-dab1-4bf9-b1a4-beec6c664021 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:30.178934402 +0000 UTC m=+10.053864459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs") pod "network-metrics-daemon-l56h5" (UID: "1c08e560-dab1-4bf9-b1a4-beec6c664021") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:26.279885 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:26.279843    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:26.280063 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.280029    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:26.280131 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.280098    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret podName:0d1d26c8-baa0-4bc3-97bf-ed62049a59f5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:30.280079653 +0000 UTC m=+10.155009713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret") pod "global-pull-secret-syncer-qp8n2" (UID: "0d1d26c8-baa0-4bc3-97bf-ed62049a59f5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:26.381171 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:26.381131    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:26.381370 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.381322    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:26.381370 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.381341    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:26.381370 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.381355    2565 projected.go:196] Error preparing data for projected volume kube-api-access-7z7q2 for pod openshift-network-diagnostics/network-check-target-9rqxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:26.381552 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.381414    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2 podName:34b1b8ba-0fdc-433a-afab-64580eb8c2b2 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:30.381396082 +0000 UTC m=+10.256326146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7z7q2" (UniqueName: "kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2") pod "network-check-target-9rqxn" (UID: "34b1b8ba-0fdc-433a-afab-64580eb8c2b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:26.596826 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:26.596741    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:26.596976 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.596869    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:26.596976 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:26.596945    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:26.597094 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:26.597025    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:27.595936 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:27.595373    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:27.595936 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:27.595538    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:28.594916 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:28.594883    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:28.595093 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:28.595018    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:28.595405 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:28.595374    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:28.595551 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:28.595507    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:29.595587 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:29.595508    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:29.596022 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:29.595625    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:30.215834 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:30.215762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:30.216023 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.215960    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:30.216072 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.216024    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs podName:1c08e560-dab1-4bf9-b1a4-beec6c664021 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:38.21600441 +0000 UTC m=+18.090934468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs") pod "network-metrics-daemon-l56h5" (UID: "1c08e560-dab1-4bf9-b1a4-beec6c664021") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:30.317429 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:30.316846    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:30.317429 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.316991    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:30.317429 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.317073    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret podName:0d1d26c8-baa0-4bc3-97bf-ed62049a59f5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:38.317050641 +0000 UTC m=+18.191980686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret") pod "global-pull-secret-syncer-qp8n2" (UID: "0d1d26c8-baa0-4bc3-97bf-ed62049a59f5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:30.418581 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:30.417929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:30.418581 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.418132    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:30.418581 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.418150    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:30.418581 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.418165    2565 projected.go:196] Error preparing data for projected volume kube-api-access-7z7q2 for pod openshift-network-diagnostics/network-check-target-9rqxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:30.418581 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.418238    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2 podName:34b1b8ba-0fdc-433a-afab-64580eb8c2b2 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:38.418218728 +0000 UTC m=+18.293148784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7z7q2" (UniqueName: "kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2") pod "network-check-target-9rqxn" (UID: "34b1b8ba-0fdc-433a-afab-64580eb8c2b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:30.596791 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:30.596226    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:30.596791 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.596359    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:30.596791 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:30.596708    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:30.597391 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:30.596826    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:31.595966 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:31.595472    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:31.595966 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:31.595599    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:32.595040 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:32.595005    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:32.595505 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:32.595145    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:32.595505 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:32.595189    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:32.595505 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:32.595285    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:33.594639 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:33.594605    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:33.594810 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:33.594703    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:34.595068 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:34.595038    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:34.595536 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:34.595074    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:34.595536 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:34.595146    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:34.595536 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:34.595250    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:35.594968 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:35.594936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:35.595159 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:35.595052    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:36.595183 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:36.595150    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:36.595608 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:36.595150    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:36.595608 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:36.595302    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:36.595608 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:36.595334    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:37.594529 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:37.594498    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:37.594717 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:37.594599    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:38.279810 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:38.279776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:38.280291 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.279936    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:38.280291 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.280017    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs podName:1c08e560-dab1-4bf9-b1a4-beec6c664021 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:54.279995269 +0000 UTC m=+34.154925316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs") pod "network-metrics-daemon-l56h5" (UID: "1c08e560-dab1-4bf9-b1a4-beec6c664021") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:38.380481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:38.380440    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:38.380655 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.380605    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:38.380718 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.380683    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret podName:0d1d26c8-baa0-4bc3-97bf-ed62049a59f5 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:54.380662218 +0000 UTC m=+34.255592279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret") pod "global-pull-secret-syncer-qp8n2" (UID: "0d1d26c8-baa0-4bc3-97bf-ed62049a59f5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:38.481034 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:38.480994    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:38.481230 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.481183    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:38.481296 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.481234    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:38.481296 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.481249    2565 projected.go:196] Error preparing data for projected volume kube-api-access-7z7q2 for pod openshift-network-diagnostics/network-check-target-9rqxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:38.481422 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.481315    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2 podName:34b1b8ba-0fdc-433a-afab-64580eb8c2b2 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:54.481297511 +0000 UTC m=+34.356227578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7z7q2" (UniqueName: "kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2") pod "network-check-target-9rqxn" (UID: "34b1b8ba-0fdc-433a-afab-64580eb8c2b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:38.595566 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:38.595489    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:38.595704 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.595626    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:38.595704 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:38.595675    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:38.595819 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:38.595799    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:39.594782 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:39.594740    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:39.595277 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:39.594875    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:40.595277 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:40.595241    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:40.595718 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:40.595348    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:40.595718 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:40.595403    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:40.595718 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:40.595524    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:41.595660 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.595505    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:41.596267 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:41.595737    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:41.696374 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.696335    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"bdf10ad2c2ed3bec7e07c11fdf4ab2c365d68567c27b6367cd08e24f6d87c55f"}
Sep 18 20:20:41.696502 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.696420    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"09455e8385377f7ded172eac784aad849dad53806d8576759145ab17aaeed211"}
Sep 18 20:20:41.696502 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.696465    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"9b01a823a65e0238ad64bc2a8e07d9237c5680b782fd1039a6083b3da7495574"}
Sep 18 20:20:41.696502 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.696496    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"1adb05fb4cfe810cda66c44ee54a856939e85b1b0a693f6e1168857b5443eb92"}
Sep 18 20:20:41.697685 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.697659    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g" event={"ID":"382faecf-06e9-449f-b0f8-7028aaabe9b5","Type":"ContainerStarted","Data":"e41c70de533aba1514e06e2e5c1c922f74e6edbcb2dd2899c7a711efcae975a6"}
Sep 18 20:20:41.699071 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.699049    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkvcr" event={"ID":"a3f61d6b-591b-47fb-9087-c422a223031f","Type":"ContainerStarted","Data":"831596e07c5778aa266b3f2812992bdf71797c463310917f1d5763f588afbace"}
Sep 18 20:20:41.700532 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.700511    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8fkmr" event={"ID":"f23a7662-58e8-41a3-80e3-bcf87e30af10","Type":"ContainerStarted","Data":"c07c9577109f107289fd04ac21fb1210a5feabb55811d48455d776cd59260da4"}
Sep 18 20:20:41.702007 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.701983    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fgtsn" event={"ID":"a723e0f7-c700-46c0-ae2a-73ce8f59dff9","Type":"ContainerStarted","Data":"d470dacc07ee87ebf2acd48c5747cc67a210cee7af0293592e4e1084b2d42268"}
Sep 18 20:20:41.703786 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.703761    2565 generic.go:356] "Generic (PLEG): container finished" podID="76292b1c-a487-4039-8548-1cadba4c8fe4" containerID="8281d25f4003564585bd32cddc8f25fb1726d719f5fde36271f104e73a8d9ec0" exitCode=0
Sep 18 20:20:41.703867 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.703832    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerDied","Data":"8281d25f4003564585bd32cddc8f25fb1726d719f5fde36271f104e73a8d9ec0"}
Sep 18 20:20:41.705520 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.705494    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkq9j" event={"ID":"6fce8f58-7f3c-402c-bd7f-0318ccf013d3","Type":"ContainerStarted","Data":"1c2f45d82b8dae846d42c91a2f57c1800b710980843772167c6c10e782e6b0e3"}
Sep 18 20:20:41.707043 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.707025    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljrz9" event={"ID":"803a9562-345e-4a66-9a13-095c205d61ee","Type":"ContainerStarted","Data":"b6bab9a4c058ca7f736450aeeeda9bb81300d99613597002778b0631d6d491b5"}
Sep 18 20:20:41.715075 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.715032    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bkvcr" podStartSLOduration=3.101593087 podStartE2EDuration="20.715021176s" totalImagesPullingTime="17.613428089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:41.714653998 +0000 UTC m=+21.589584063" watchObservedRunningTime="2026-09-18 20:20:41.715021176 +0000 UTC m=+21.589951240"
Sep 18 20:20:41.715150 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.715119    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-48.ec2.internal" podStartSLOduration=20.715115636 podStartE2EDuration="20.715115636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:25.687075541 +0000 UTC m=+5.562005626" watchObservedRunningTime="2026-09-18 20:20:41.715115636 +0000 UTC m=+21.590045701"
Sep 18 20:20:41.735988 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.735945    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ljrz9" podStartSLOduration=4.056282367 podStartE2EDuration="21.735932943s" totalImagesPullingTime="17.679650576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:41.735339229 +0000 UTC m=+21.610269294" watchObservedRunningTime="2026-09-18 20:20:41.735932943 +0000 UTC m=+21.610863006"
Sep 18 20:20:41.756654 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.756617    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8fkmr" podStartSLOduration=4.13636516 podStartE2EDuration="21.756605474s" totalImagesPullingTime="17.620240314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:41.75637747 +0000 UTC m=+21.631307535" watchObservedRunningTime="2026-09-18 20:20:41.756605474 +0000 UTC m=+21.631535538"
Sep 18 20:20:41.812338 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.812288    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fgtsn" podStartSLOduration=10.575734978 podStartE2EDuration="20.812270908s" totalImagesPullingTime="10.23653593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:41.79337016 +0000 UTC m=+21.668300225" watchObservedRunningTime="2026-09-18 20:20:41.812270908 +0000 UTC m=+21.687200972"
Sep 18 20:20:41.814169 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:41.812989    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lkq9j" podStartSLOduration=4.189033086 podStartE2EDuration="21.812972762s" totalImagesPullingTime="17.623939676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:41.811738973 +0000 UTC m=+21.686669031" watchObservedRunningTime="2026-09-18 20:20:41.812972762 +0000 UTC m=+21.687902826"
Sep 18 20:20:42.595373 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.595331    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:42.595519 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.595339    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:42.595519 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:42.595459    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:42.595632 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:42.595552    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:42.627430 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.627400    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 20:20:42.713159 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.713120    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"a1483762ed7b092b11898330829ca395634032fb93953cecfc032148d3c498d3"}
Sep 18 20:20:42.713369 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.713236    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"394f1bae87947709ea8cbc9cd0cf6eec63908f0f9e2e3f1f7701d988002b80d9"}
Sep 18 20:20:42.715154 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.715129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g" event={"ID":"382faecf-06e9-449f-b0f8-7028aaabe9b5","Type":"ContainerStarted","Data":"dcf7b6ec08e868302ecd8849bef27e3bb81bff1265ea71010302e267e605bd26"}
Sep 18 20:20:42.716972 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.716942    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9zng" event={"ID":"0e45d494-de53-40cb-9602-aa50960e4ca8","Type":"ContainerStarted","Data":"7f92974cbbd27d6f00bac23e7beb3ef1460cff4fdbdb25647a97d5cd5e343b93"}
Sep 18 20:20:42.731794 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:42.731709    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c9zng" podStartSLOduration=4.11393333 podStartE2EDuration="21.731692081s" totalImagesPullingTime="17.617758751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:42.731434524 +0000 UTC m=+22.606364612" watchObservedRunningTime="2026-09-18 20:20:42.731692081 +0000 UTC m=+22.606622145"
Sep 18 20:20:43.593616 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:43.593467    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T20:20:42.627423586Z","UUID":"2aafe20b-273a-4d7a-92e2-80b9e6aa698b","Handler":null,"Name":"","Endpoint":""}
Sep 18 20:20:43.594605 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:43.594573    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:43.594752 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:43.594670    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:43.595084 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:43.595062    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 20:20:43.595197 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:43.595105    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 20:20:44.594737 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:44.594704    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:44.595275 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:44.594716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:44.595275 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:44.594835    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:44.595275 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:44.594912    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:44.725806 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:44.725766    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"ac8b804a7fe4ba719eb44214c192e666a7f1e8500967b2116a12435e74701a89"}
Sep 18 20:20:44.728095 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:44.728070    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g" event={"ID":"382faecf-06e9-449f-b0f8-7028aaabe9b5","Type":"ContainerStarted","Data":"07e04c7a0f79008158a2c412bad7284aba15b998fc74cf1f10986b16915129f8"}
Sep 18 20:20:44.756736 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:44.756692    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9td4g" podStartSLOduration=5.025469573 podStartE2EDuration="24.756676243s" totalImagesPullingTime="19.73120667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:44.756199104 +0000 UTC m=+24.631129222" watchObservedRunningTime="2026-09-18 20:20:44.756676243 +0000 UTC m=+24.631606310"
Sep 18 20:20:45.595128 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:45.595098    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:45.595498 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:45.595195    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:45.778579 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:45.778544    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:45.779123 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:45.779104    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:46.595246 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.595070    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:46.596107 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.595106    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:46.596107 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:46.595324    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:46.596107 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:46.595420    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:46.735823 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.735789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" event={"ID":"85eb79b8-f47b-40da-a41c-94b053ddc4a3","Type":"ContainerStarted","Data":"de45962c66dcb591b3e2cf9502e59766bc54f9612c6d207a5c5df5ee32f55144"}
Sep 18 20:20:46.736044 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.736024    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:46.737767 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.737737    2565 generic.go:356] "Generic (PLEG): container finished" podID="76292b1c-a487-4039-8548-1cadba4c8fe4" containerID="2aab2d3d5a4c9659fbfeb795fbefaf6cc47b3920522022e910d62bc1fc22bf99" exitCode=0
Sep 18 20:20:46.737866 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.737818    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerDied","Data":"2aab2d3d5a4c9659fbfeb795fbefaf6cc47b3920522022e910d62bc1fc22bf99"}
Sep 18 20:20:46.751505 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.751485    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:46.778721 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:46.778676    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj" podStartSLOduration=9.100682379 podStartE2EDuration="26.778660884s" totalImagesPullingTime="17.677978505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:46.778044013 +0000 UTC m=+26.652974096" watchObservedRunningTime="2026-09-18 20:20:46.778660884 +0000 UTC m=+26.653590955"
Sep 18 20:20:47.595546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.595371    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:47.595928 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:47.595660    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:47.740996 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.740955    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:47.741142 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.741047    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:47.756479 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.756453    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:20:47.995916 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.995888    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qp8n2"]
Sep 18 20:20:47.996031 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.995992    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:47.996092 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:47.996071    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:47.997600 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.997574    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l56h5"]
Sep 18 20:20:47.997705 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.997690    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:47.997814 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:47.997794    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:47.998292 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.998254    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9rqxn"]
Sep 18 20:20:47.998390 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:47.998334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:47.998453 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:47.998421    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:48.744893 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:48.744861    2565 generic.go:356] "Generic (PLEG): container finished" podID="76292b1c-a487-4039-8548-1cadba4c8fe4" containerID="1166fca6b7bf748f0a520c6f0bf7988186ffcd2b1d93bdebfe986e75243a3205" exitCode=0
Sep 18 20:20:48.745330 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:48.744955    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerDied","Data":"1166fca6b7bf748f0a520c6f0bf7988186ffcd2b1d93bdebfe986e75243a3205"}
Sep 18 20:20:49.595172 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:49.595140    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:49.595474 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:49.595139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:49.595474 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:49.595280    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:49.595474 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:49.595139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:49.595474 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:49.595343    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:49.595474 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:49.595415    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:50.196104 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:50.196069    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:50.196529 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:50.196235    2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 20:20:50.196728 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:50.196711    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fgtsn"
Sep 18 20:20:50.753568 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:50.753383    2565 generic.go:356] "Generic (PLEG): container finished" podID="76292b1c-a487-4039-8548-1cadba4c8fe4" containerID="0a7ddb23c416fed21a80c553470c056c900558e788a72aed6cf5e2a0da24113d" exitCode=0
Sep 18 20:20:50.753680 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:50.753588    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerDied","Data":"0a7ddb23c416fed21a80c553470c056c900558e788a72aed6cf5e2a0da24113d"}
Sep 18 20:20:51.595510 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:51.595329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:51.595925 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:51.595356    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:51.595925 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:51.595595    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:51.595925 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:51.595685    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:51.595925 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:51.595356    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:51.595925 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:51.595773    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:53.595080 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:53.595044    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:53.595525 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:53.595093    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:53.595525 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:53.595093    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:53.595525 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:53.595165    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rqxn" podUID="34b1b8ba-0fdc-433a-afab-64580eb8c2b2"
Sep 18 20:20:53.595525 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:53.595274    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qp8n2" podUID="0d1d26c8-baa0-4bc3-97bf-ed62049a59f5"
Sep 18 20:20:53.595525 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:53.595360    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l56h5" podUID="1c08e560-dab1-4bf9-b1a4-beec6c664021"
Sep 18 20:20:53.972988 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:53.972960    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-48.ec2.internal" event="NodeReady"
Sep 18 20:20:53.973168 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:53.973103    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 20:20:54.008283 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.008237    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"]
Sep 18 20:20:54.014156 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.014127    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.016909 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.016880    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pr778\""
Sep 18 20:20:54.017386 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.017361    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 20:20:54.017386 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.017372    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 20:20:54.017386 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.017386    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 20:20:54.022488 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.022464    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 20:20:54.024168 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.024147    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6gc77"]
Sep 18 20:20:54.028148 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.028120    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.029546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.029514    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"]
Sep 18 20:20:54.030078 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.030062    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c8cpg"]
Sep 18 20:20:54.030439 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.030415    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 20:20:54.030439 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.030417    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 20:20:54.030768 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.030752    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmtgt\""
Sep 18 20:20:54.033852 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.033831    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.035877 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.035857    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 20:20:54.035877 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.035868    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 20:20:54.036026 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.036003    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jl6dr\""
Sep 18 20:20:54.036090 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.036046    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 20:20:54.037990 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.037862    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6gc77"]
Sep 18 20:20:54.042713 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.042625    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c8cpg"]
Sep 18 20:20:54.093428 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093397    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34fb603c-9f1a-46be-9e79-6f6a4f48527b-config-volume\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.093594 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093434    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23831892-77d6-4bcf-842f-074248bc8db6-ca-trust-extracted\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.093594 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fb603c-9f1a-46be-9e79-6f6a4f48527b-tmp-dir\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.093594 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093493    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.093713 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093596    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.093713 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093634    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-bound-sa-token\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.093713 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093676    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-trusted-ca\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.093713 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-installation-pull-secrets\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.093863 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093760    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfgdw\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-kube-api-access-rfgdw\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.093863 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093779    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7p48\" (UniqueName: \"kubernetes.io/projected/34fb603c-9f1a-46be-9e79-6f6a4f48527b-kube-api-access-v7p48\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.093863 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-registry-certificates\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.093863 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.093821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-image-registry-private-configuration\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.194735 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.194696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-bound-sa-token\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.194908 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.194759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-trusted-ca\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.194908 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.194784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-installation-pull-secrets\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.194908 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.194837    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfgdw\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-kube-api-access-rfgdw\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.195078 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.194955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d42f20c5-56ec-43f3-9d04-588f1608092e-cert\") pod \"ingress-canary-c8cpg\" (UID: \"d42f20c5-56ec-43f3-9d04-588f1608092e\") " pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.195078 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195020    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7p48\" (UniqueName: \"kubernetes.io/projected/34fb603c-9f1a-46be-9e79-6f6a4f48527b-kube-api-access-v7p48\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.195078 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195053    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-registry-certificates\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.195237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-image-registry-private-configuration\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.195237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7jkw\" (UniqueName: \"kubernetes.io/projected/d42f20c5-56ec-43f3-9d04-588f1608092e-kube-api-access-t7jkw\") pod \"ingress-canary-c8cpg\" (UID: \"d42f20c5-56ec-43f3-9d04-588f1608092e\") " pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.195237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34fb603c-9f1a-46be-9e79-6f6a4f48527b-config-volume\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.195237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195173    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23831892-77d6-4bcf-842f-074248bc8db6-ca-trust-extracted\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.195237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195201    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fb603c-9f1a-46be-9e79-6f6a4f48527b-tmp-dir\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.195453 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195247    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.195453 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195297    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.195453 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.195438    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:20:54.195596 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.195508    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls podName:34fb603c-9f1a-46be-9e79-6f6a4f48527b nodeName:}" failed. No retries permitted until 2026-09-18 20:20:54.695486748 +0000 UTC m=+34.570416805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls") pod "dns-default-6gc77" (UID: "34fb603c-9f1a-46be-9e79-6f6a4f48527b") : secret "dns-default-metrics-tls" not found
Sep 18 20:20:54.195654 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195638    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-registry-certificates\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.195735 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195714    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34fb603c-9f1a-46be-9e79-6f6a4f48527b-tmp-dir\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.195791 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.195740    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:20:54.195791 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.195754    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb7fbb4df-xt2cv: secret "image-registry-tls" not found
Sep 18 20:20:54.195865 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.195803    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls podName:23831892-77d6-4bcf-842f-074248bc8db6 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:54.69578736 +0000 UTC m=+34.570717416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls") pod "image-registry-5bb7fbb4df-xt2cv" (UID: "23831892-77d6-4bcf-842f-074248bc8db6") : secret "image-registry-tls" not found
Sep 18 20:20:54.195865 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34fb603c-9f1a-46be-9e79-6f6a4f48527b-config-volume\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.195957 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.195864    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-trusted-ca\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.196044 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.196009    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23831892-77d6-4bcf-842f-074248bc8db6-ca-trust-extracted\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.199412 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.199389    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-image-registry-private-configuration\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.199412 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.199400    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-installation-pull-secrets\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.204483 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.204460    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7p48\" (UniqueName: \"kubernetes.io/projected/34fb603c-9f1a-46be-9e79-6f6a4f48527b-kube-api-access-v7p48\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.204601 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.204466    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-bound-sa-token\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.205188 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.205154    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfgdw\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-kube-api-access-rfgdw\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.296472 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.296389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:54.296472 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.296432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d42f20c5-56ec-43f3-9d04-588f1608092e-cert\") pod \"ingress-canary-c8cpg\" (UID: \"d42f20c5-56ec-43f3-9d04-588f1608092e\") " pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.296472 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.296463    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7jkw\" (UniqueName: \"kubernetes.io/projected/d42f20c5-56ec-43f3-9d04-588f1608092e-kube-api-access-t7jkw\") pod \"ingress-canary-c8cpg\" (UID: \"d42f20c5-56ec-43f3-9d04-588f1608092e\") " pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.296746 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.296563    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:54.296746 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.296640    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs podName:1c08e560-dab1-4bf9-b1a4-beec6c664021 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:26.296621576 +0000 UTC m=+66.171551626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs") pod "network-metrics-daemon-l56h5" (UID: "1c08e560-dab1-4bf9-b1a4-beec6c664021") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:20:54.299012 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.298991    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d42f20c5-56ec-43f3-9d04-588f1608092e-cert\") pod \"ingress-canary-c8cpg\" (UID: \"d42f20c5-56ec-43f3-9d04-588f1608092e\") " pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.308257 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.308234    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7jkw\" (UniqueName: \"kubernetes.io/projected/d42f20c5-56ec-43f3-9d04-588f1608092e-kube-api-access-t7jkw\") pod \"ingress-canary-c8cpg\" (UID: \"d42f20c5-56ec-43f3-9d04-588f1608092e\") " pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.346964 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.346925    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c8cpg"
Sep 18 20:20:54.397623 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.397586    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:54.397790 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.397769    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:54.397856 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.397846    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret podName:0d1d26c8-baa0-4bc3-97bf-ed62049a59f5 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:26.397825799 +0000 UTC m=+66.272755898 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret") pod "global-pull-secret-syncer-qp8n2" (UID: "0d1d26c8-baa0-4bc3-97bf-ed62049a59f5") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:20:54.498464 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.498431    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:54.498661 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.498631    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:20:54.498661 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.498658    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:20:54.498769 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.498672    2565 projected.go:196] Error preparing data for projected volume kube-api-access-7z7q2 for pod openshift-network-diagnostics/network-check-target-9rqxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:54.498769 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.498737    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2 podName:34b1b8ba-0fdc-433a-afab-64580eb8c2b2 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:26.498719719 +0000 UTC m=+66.373649784 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7z7q2" (UniqueName: "kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2") pod "network-check-target-9rqxn" (UID: "34b1b8ba-0fdc-433a-afab-64580eb8c2b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:20:54.699922 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.699890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:54.700532 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.699963    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:54.700532 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.700007    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:20:54.700532 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.700044    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:20:54.700532 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.700055    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb7fbb4df-xt2cv: secret "image-registry-tls" not found
Sep 18 20:20:54.700532 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.700075    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls podName:34fb603c-9f1a-46be-9e79-6f6a4f48527b nodeName:}" failed. No retries permitted until 2026-09-18 20:20:55.700055549 +0000 UTC m=+35.574985596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls") pod "dns-default-6gc77" (UID: "34fb603c-9f1a-46be-9e79-6f6a4f48527b") : secret "dns-default-metrics-tls" not found
Sep 18 20:20:54.700532 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:54.700091    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls podName:23831892-77d6-4bcf-842f-074248bc8db6 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:55.700081292 +0000 UTC m=+35.575011362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls") pod "image-registry-5bb7fbb4df-xt2cv" (UID: "23831892-77d6-4bcf-842f-074248bc8db6") : secret "image-registry-tls" not found
Sep 18 20:20:54.953011 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:54.952747    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c8cpg"]
Sep 18 20:20:54.966019 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:20:54.965989    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42f20c5_56ec_43f3_9d04_588f1608092e.slice/crio-6e82f2e14171d1991d352c40dbf4b9a967074d188805506dbd7b33f8e95c4786 WatchSource:0}: Error finding container 6e82f2e14171d1991d352c40dbf4b9a967074d188805506dbd7b33f8e95c4786: Status 404 returned error can't find the container with id 6e82f2e14171d1991d352c40dbf4b9a967074d188805506dbd7b33f8e95c4786
Sep 18 20:20:55.595417 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.595380    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:20:55.595586 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.595393    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:20:55.595586 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.595513    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:20:55.600647 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.600622    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:20:55.601245 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.601224    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 20:20:55.601375 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.601254    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:20:55.601375 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.601224    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 20:20:55.601375 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.601224    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27fhx\""
Sep 18 20:20:55.601564 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.601531    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmjzr\""
Sep 18 20:20:55.709065 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.709035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:55.709741 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.709089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:55.709741 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:55.709227    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:20:55.709741 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:55.709234    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:20:55.709741 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:55.709255    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb7fbb4df-xt2cv: secret "image-registry-tls" not found
Sep 18 20:20:55.709741 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:55.709285    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls podName:34fb603c-9f1a-46be-9e79-6f6a4f48527b nodeName:}" failed. No retries permitted until 2026-09-18 20:20:57.709268377 +0000 UTC m=+37.584198438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls") pod "dns-default-6gc77" (UID: "34fb603c-9f1a-46be-9e79-6f6a4f48527b") : secret "dns-default-metrics-tls" not found
Sep 18 20:20:55.709741 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:55.709310    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls podName:23831892-77d6-4bcf-842f-074248bc8db6 nodeName:}" failed. No retries permitted until 2026-09-18 20:20:57.709292393 +0000 UTC m=+37.584222453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls") pod "image-registry-5bb7fbb4df-xt2cv" (UID: "23831892-77d6-4bcf-842f-074248bc8db6") : secret "image-registry-tls" not found
Sep 18 20:20:55.770693 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.770661    2565 generic.go:356] "Generic (PLEG): container finished" podID="76292b1c-a487-4039-8548-1cadba4c8fe4" containerID="a6c8396f9dfa3c4ff0cdd6bd4f2dc470bc9e842ad30c6fba2e1883ce0de84667" exitCode=0
Sep 18 20:20:55.770862 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.770748    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerDied","Data":"a6c8396f9dfa3c4ff0cdd6bd4f2dc470bc9e842ad30c6fba2e1883ce0de84667"}
Sep 18 20:20:55.772071 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:55.772044    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c8cpg" event={"ID":"d42f20c5-56ec-43f3-9d04-588f1608092e","Type":"ContainerStarted","Data":"6e82f2e14171d1991d352c40dbf4b9a967074d188805506dbd7b33f8e95c4786"}
Sep 18 20:20:56.778440 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:56.778404    2565 generic.go:356] "Generic (PLEG): container finished" podID="76292b1c-a487-4039-8548-1cadba4c8fe4" containerID="a7673cad3f301e747df44a5a3492b1a1d2910bf28f80e50cb8a590c915ad5cd9" exitCode=0
Sep 18 20:20:56.778913 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:56.778479    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerDied","Data":"a7673cad3f301e747df44a5a3492b1a1d2910bf28f80e50cb8a590c915ad5cd9"}
Sep 18 20:20:57.725012 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:57.724974    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:20:57.725259 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:57.725038    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:20:57.725259 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:57.725134    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:20:57.725259 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:57.725160    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb7fbb4df-xt2cv: secret "image-registry-tls" not found
Sep 18 20:20:57.725259 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:57.725141    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:20:57.725419 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:57.725235    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls podName:23831892-77d6-4bcf-842f-074248bc8db6 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:01.72519995 +0000 UTC m=+41.600130018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls") pod "image-registry-5bb7fbb4df-xt2cv" (UID: "23831892-77d6-4bcf-842f-074248bc8db6") : secret "image-registry-tls" not found
Sep 18 20:20:57.725419 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:20:57.725299    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls podName:34fb603c-9f1a-46be-9e79-6f6a4f48527b nodeName:}" failed. No retries permitted until 2026-09-18 20:21:01.725282331 +0000 UTC m=+41.600212372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls") pod "dns-default-6gc77" (UID: "34fb603c-9f1a-46be-9e79-6f6a4f48527b") : secret "dns-default-metrics-tls" not found
Sep 18 20:20:57.783668 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:57.783637    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xznrr" event={"ID":"76292b1c-a487-4039-8548-1cadba4c8fe4","Type":"ContainerStarted","Data":"aad42a66c59bf4cbffe6b92f8a87a04433c775e78190480be90703efe8a56fcb"}
Sep 18 20:20:57.785033 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:57.785002    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c8cpg" event={"ID":"d42f20c5-56ec-43f3-9d04-588f1608092e","Type":"ContainerStarted","Data":"d545214b6b0d5e1d89d80fcd5914c7e8dbb03b1224e8f1343ff45fc948dedf04"}
Sep 18 20:20:57.808603 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:57.808548    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xznrr" podStartSLOduration=9.769399741 podStartE2EDuration="37.808531735s" totalImagesPullingTime="28.039131994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:57.80786507 +0000 UTC m=+37.682795133" watchObservedRunningTime="2026-09-18 20:20:57.808531735 +0000 UTC m=+37.683461801"
Sep 18 20:20:57.823815 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:20:57.823772    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c8cpg" podStartSLOduration=1.684465936 podStartE2EDuration="3.823761758s" totalImagesPullingTime="2.139295822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:57.823256813 +0000 UTC m=+37.698186893" watchObservedRunningTime="2026-09-18 20:20:57.823761758 +0000 UTC m=+37.698691823"
Sep 18 20:21:01.755895 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:01.755850    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:21:01.756315 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:01.755922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:21:01.756315 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:01.755966    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:21:01.756315 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:01.755989    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb7fbb4df-xt2cv: secret "image-registry-tls" not found
Sep 18 20:21:01.756315 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:01.756045    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls podName:23831892-77d6-4bcf-842f-074248bc8db6 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:09.756029753 +0000 UTC m=+49.630959812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls") pod "image-registry-5bb7fbb4df-xt2cv" (UID: "23831892-77d6-4bcf-842f-074248bc8db6") : secret "image-registry-tls" not found
Sep 18 20:21:01.756315 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:01.756046    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:21:01.756315 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:01.756086    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls podName:34fb603c-9f1a-46be-9e79-6f6a4f48527b nodeName:}" failed. No retries permitted until 2026-09-18 20:21:09.756074687 +0000 UTC m=+49.631004752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls") pod "dns-default-6gc77" (UID: "34fb603c-9f1a-46be-9e79-6f6a4f48527b") : secret "dns-default-metrics-tls" not found
Sep 18 20:21:03.513351 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.513312    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"]
Sep 18 20:21:03.545580 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.545551    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"]
Sep 18 20:21:03.545726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.545674    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.548772 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.548571    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 18 20:21:03.548772 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.548637    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 18 20:21:03.548772 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.548659    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kjvtw\""
Sep 18 20:21:03.548772 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.548665    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:21:03.548772 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.548683    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 18 20:21:03.559538 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.559520    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-x94rj"]
Sep 18 20:21:03.559643 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.559627    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:03.561850 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.561833    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 18 20:21:03.562232 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.562196    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tsgx8\""
Sep 18 20:21:03.562279 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.562225    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 18 20:21:03.562370 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.562353    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:21:03.581566 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.581543    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"]
Sep 18 20:21:03.581646 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.581578    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"]
Sep 18 20:21:03.581646 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.581606    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-x94rj"]
Sep 18 20:21:03.581646 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.581618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.583483 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.583464    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:21:03.583591 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.583491    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mnwkt\""
Sep 18 20:21:03.583591 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.583573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 18 20:21:03.583856 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.583840    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 18 20:21:03.584434 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.584396    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 18 20:21:03.588999 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.588977    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 18 20:21:03.668242 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668196    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kwm6\" (UniqueName: \"kubernetes.io/projected/6f534f4e-ec58-41c6-a920-68a79947bdbc-kube-api-access-6kwm6\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.668242 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668245    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f534f4e-ec58-41c6-a920-68a79947bdbc-serving-cert\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.668484 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f534f4e-ec58-41c6-a920-68a79947bdbc-config\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.668484 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668295    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e36a61f-4a55-4981-be3c-b23fce9b617e-config\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.668484 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668338    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2rw9\" (UniqueName: \"kubernetes.io/projected/4e36a61f-4a55-4981-be3c-b23fce9b617e-kube-api-access-w2rw9\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.668484 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668435    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dp94\" (UniqueName: \"kubernetes.io/projected/df51014c-f4e0-460c-84a0-007bc6268194-kube-api-access-5dp94\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:03.668484 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668469    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f534f4e-ec58-41c6-a920-68a79947bdbc-trusted-ca\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.668690 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668503    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:03.668690 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.668528    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e36a61f-4a55-4981-be3c-b23fce9b617e-serving-cert\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.768913 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.768831    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:03.768913 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.768869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e36a61f-4a55-4981-be3c-b23fce9b617e-serving-cert\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.769139 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.768927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kwm6\" (UniqueName: \"kubernetes.io/projected/6f534f4e-ec58-41c6-a920-68a79947bdbc-kube-api-access-6kwm6\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.769139 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.768953    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f534f4e-ec58-41c6-a920-68a79947bdbc-serving-cert\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.769139 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.768986    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f534f4e-ec58-41c6-a920-68a79947bdbc-config\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.769139 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:03.768987    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:21:03.769139 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:03.769073    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls podName:df51014c-f4e0-460c-84a0-007bc6268194 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:04.269055362 +0000 UTC m=+44.143985417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-78q94" (UID: "df51014c-f4e0-460c-84a0-007bc6268194") : secret "samples-operator-tls" not found
Sep 18 20:21:03.769139 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.769112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e36a61f-4a55-4981-be3c-b23fce9b617e-config\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.769481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.769152    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2rw9\" (UniqueName: \"kubernetes.io/projected/4e36a61f-4a55-4981-be3c-b23fce9b617e-kube-api-access-w2rw9\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.769481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.769248    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dp94\" (UniqueName: \"kubernetes.io/projected/df51014c-f4e0-460c-84a0-007bc6268194-kube-api-access-5dp94\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:03.769481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.769279    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f534f4e-ec58-41c6-a920-68a79947bdbc-trusted-ca\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.773255 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.770105    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e36a61f-4a55-4981-be3c-b23fce9b617e-config\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.773255 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.770105    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f534f4e-ec58-41c6-a920-68a79947bdbc-config\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.773255 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.772331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f534f4e-ec58-41c6-a920-68a79947bdbc-trusted-ca\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.773638 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.773608    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e36a61f-4a55-4981-be3c-b23fce9b617e-serving-cert\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.774421 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.774398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f534f4e-ec58-41c6-a920-68a79947bdbc-serving-cert\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.786683 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.786658    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2rw9\" (UniqueName: \"kubernetes.io/projected/4e36a61f-4a55-4981-be3c-b23fce9b617e-kube-api-access-w2rw9\") pod \"service-ca-operator-57cdc4b768-6q2ll\" (UID: \"4e36a61f-4a55-4981-be3c-b23fce9b617e\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.786790 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.786725    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dp94\" (UniqueName: \"kubernetes.io/projected/df51014c-f4e0-460c-84a0-007bc6268194-kube-api-access-5dp94\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:03.788220 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.788192    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kwm6\" (UniqueName: \"kubernetes.io/projected/6f534f4e-ec58-41c6-a920-68a79947bdbc-kube-api-access-6kwm6\") pod \"console-operator-fd4bb46c-x94rj\" (UID: \"6f534f4e-ec58-41c6-a920-68a79947bdbc\") " pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:03.854832 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.854803    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"
Sep 18 20:21:03.890556 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:03.890528    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:04.156765 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:04.156694    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll"]
Sep 18 20:21:04.160030 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:04.160009    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e36a61f_4a55_4981_be3c_b23fce9b617e.slice/crio-8c9848ead4f444f1430c66d239ce7739066997691c8e3a924a85f5d9dcfb6cd0 WatchSource:0}: Error finding container 8c9848ead4f444f1430c66d239ce7739066997691c8e3a924a85f5d9dcfb6cd0: Status 404 returned error can't find the container with id 8c9848ead4f444f1430c66d239ce7739066997691c8e3a924a85f5d9dcfb6cd0
Sep 18 20:21:04.190118 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:04.190089    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-x94rj"]
Sep 18 20:21:04.193061 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:04.193029    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f534f4e_ec58_41c6_a920_68a79947bdbc.slice/crio-ffccc6a67b22c12323be37d625fa8241c448f6f8090eb7fcbd9ccc19e44fa1f0 WatchSource:0}: Error finding container ffccc6a67b22c12323be37d625fa8241c448f6f8090eb7fcbd9ccc19e44fa1f0: Status 404 returned error can't find the container with id ffccc6a67b22c12323be37d625fa8241c448f6f8090eb7fcbd9ccc19e44fa1f0
Sep 18 20:21:04.273076 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:04.273039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:04.273225 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:04.273194    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:21:04.273286 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:04.273276    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls podName:df51014c-f4e0-460c-84a0-007bc6268194 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:05.273259765 +0000 UTC m=+45.148189822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-78q94" (UID: "df51014c-f4e0-460c-84a0-007bc6268194") : secret "samples-operator-tls" not found
Sep 18 20:21:04.801694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:04.801647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" event={"ID":"6f534f4e-ec58-41c6-a920-68a79947bdbc","Type":"ContainerStarted","Data":"ffccc6a67b22c12323be37d625fa8241c448f6f8090eb7fcbd9ccc19e44fa1f0"}
Sep 18 20:21:04.802955 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:04.802929    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll" event={"ID":"4e36a61f-4a55-4981-be3c-b23fce9b617e","Type":"ContainerStarted","Data":"8c9848ead4f444f1430c66d239ce7739066997691c8e3a924a85f5d9dcfb6cd0"}
Sep 18 20:21:05.280411 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:05.280220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:05.280411 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:05.280391    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:21:05.280740 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:05.280475    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls podName:df51014c-f4e0-460c-84a0-007bc6268194 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:07.280452483 +0000 UTC m=+47.155382540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-78q94" (UID: "df51014c-f4e0-460c-84a0-007bc6268194") : secret "samples-operator-tls" not found
Sep 18 20:21:07.297942 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:07.297854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:07.298315 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:07.297988    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:21:07.298315 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:07.298046    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls podName:df51014c-f4e0-460c-84a0-007bc6268194 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:11.298031582 +0000 UTC m=+51.172961637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-78q94" (UID: "df51014c-f4e0-460c-84a0-007bc6268194") : secret "samples-operator-tls" not found
Sep 18 20:21:07.811270 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:07.811230    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll" event={"ID":"4e36a61f-4a55-4981-be3c-b23fce9b617e","Type":"ContainerStarted","Data":"307e1a061d016c34aa589fd4b9b32ddd22c71a4ee1ce8663aad8be29ad017b8f"}
Sep 18 20:21:07.812880 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:07.812861    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/0.log"
Sep 18 20:21:07.813025 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:07.812898    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f534f4e-ec58-41c6-a920-68a79947bdbc" containerID="d52dad280cb0f2e25e96768da2612ad08cbc56e4c52d28db12cd6af9b92ac7c0" exitCode=255
Sep 18 20:21:07.813025 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:07.812928    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" event={"ID":"6f534f4e-ec58-41c6-a920-68a79947bdbc","Type":"ContainerDied","Data":"d52dad280cb0f2e25e96768da2612ad08cbc56e4c52d28db12cd6af9b92ac7c0"}
Sep 18 20:21:07.813269 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:07.813254    2565 scope.go:117] "RemoveContainer" containerID="d52dad280cb0f2e25e96768da2612ad08cbc56e4c52d28db12cd6af9b92ac7c0"
Sep 18 20:21:07.833910 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:07.833868    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll" podStartSLOduration=1.939880918 podStartE2EDuration="4.8338548s" totalImagesPullingTime="2.893973882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:07.833235985 +0000 UTC m=+47.708166068" watchObservedRunningTime="2026-09-18 20:21:07.8338548 +0000 UTC m=+47.708784896"
Sep 18 20:21:08.717126 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.717088    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pd9m9"]
Sep 18 20:21:08.732588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.732563    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:08.734806 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.734782    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-299f5\""
Sep 18 20:21:08.734938 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.734786    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 20:21:08.735369 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.735352    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 20:21:08.735369 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.735363    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 20:21:08.735482 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.735377    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 20:21:08.745357 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.745334    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pd9m9"]
Sep 18 20:21:08.817264 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.817235    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:21:08.817673 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.817657    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/0.log"
Sep 18 20:21:08.817763 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.817692    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f534f4e-ec58-41c6-a920-68a79947bdbc" containerID="d6f4a98df1fbdf6655d61385a2137e3879e8d653d5f9933182fa55ba4251127d" exitCode=255
Sep 18 20:21:08.817763 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.817722    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" event={"ID":"6f534f4e-ec58-41c6-a920-68a79947bdbc","Type":"ContainerDied","Data":"d6f4a98df1fbdf6655d61385a2137e3879e8d653d5f9933182fa55ba4251127d"}
Sep 18 20:21:08.817869 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.817773    2565 scope.go:117] "RemoveContainer" containerID="d52dad280cb0f2e25e96768da2612ad08cbc56e4c52d28db12cd6af9b92ac7c0"
Sep 18 20:21:08.818039 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.818023    2565 scope.go:117] "RemoveContainer" containerID="d6f4a98df1fbdf6655d61385a2137e3879e8d653d5f9933182fa55ba4251127d"
Sep 18 20:21:08.818269 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:08.818249    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-x94rj_openshift-console-operator(6f534f4e-ec58-41c6-a920-68a79947bdbc)\"" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" podUID="6f534f4e-ec58-41c6-a920-68a79947bdbc"
Sep 18 20:21:08.909127 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.909085    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d5039-6364-4256-bb67-b8f2fd3b336e-data-volume\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:08.909463 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.909440    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a45d5039-6364-4256-bb67-b8f2fd3b336e-crio-socket\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:08.909545 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.909490    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:08.909608 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.909584    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:08.909739 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:08.909724    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmgbf\" (UniqueName: \"kubernetes.io/projected/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-api-access-vmgbf\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.010835 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.010746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a45d5039-6364-4256-bb67-b8f2fd3b336e-crio-socket\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.010835 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.010790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.011034 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.010838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.011034 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.010896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmgbf\" (UniqueName: \"kubernetes.io/projected/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-api-access-vmgbf\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.011034 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.010939    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d5039-6364-4256-bb67-b8f2fd3b336e-data-volume\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.011034 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.010973    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:09.011165 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.010980    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a45d5039-6364-4256-bb67-b8f2fd3b336e-crio-socket\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.011165 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.010980    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:21:09.011165 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.011132    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls podName:a45d5039-6364-4256-bb67-b8f2fd3b336e nodeName:}" failed. No retries permitted until 2026-09-18 20:21:09.511117983 +0000 UTC m=+49.386048025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pd9m9" (UID: "a45d5039-6364-4256-bb67-b8f2fd3b336e") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:09.011322 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.011186    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm podName:a45d5039-6364-4256-bb67-b8f2fd3b336e nodeName:}" failed. No retries permitted until 2026-09-18 20:21:09.511167448 +0000 UTC m=+49.386097505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pd9m9" (UID: "a45d5039-6364-4256-bb67-b8f2fd3b336e") : configmap "kube-rbac-proxy" not found
Sep 18 20:21:09.011379 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.011336    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d5039-6364-4256-bb67-b8f2fd3b336e-data-volume\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.020589 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.020563    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmgbf\" (UniqueName: \"kubernetes.io/projected/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-api-access-vmgbf\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.515647 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.515606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.515647 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.515655    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:09.515870 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.515725    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:21:09.515870 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.515761    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:09.515870 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.515783    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm podName:a45d5039-6364-4256-bb67-b8f2fd3b336e nodeName:}" failed. No retries permitted until 2026-09-18 20:21:10.515771881 +0000 UTC m=+50.390701922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pd9m9" (UID: "a45d5039-6364-4256-bb67-b8f2fd3b336e") : configmap "kube-rbac-proxy" not found
Sep 18 20:21:09.515870 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.515830    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls podName:a45d5039-6364-4256-bb67-b8f2fd3b336e nodeName:}" failed. No retries permitted until 2026-09-18 20:21:10.515815865 +0000 UTC m=+50.390745931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pd9m9" (UID: "a45d5039-6364-4256-bb67-b8f2fd3b336e") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:09.818261 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.818168    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:21:09.818261 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.818243    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:21:09.818666 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.818338    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:21:09.818666 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.818363    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb7fbb4df-xt2cv: secret "image-registry-tls" not found
Sep 18 20:21:09.818666 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.818387    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:21:09.818666 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.818430    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls podName:23831892-77d6-4bcf-842f-074248bc8db6 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:25.818410995 +0000 UTC m=+65.693341061 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls") pod "image-registry-5bb7fbb4df-xt2cv" (UID: "23831892-77d6-4bcf-842f-074248bc8db6") : secret "image-registry-tls" not found
Sep 18 20:21:09.818666 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.818448    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls podName:34fb603c-9f1a-46be-9e79-6f6a4f48527b nodeName:}" failed. No retries permitted until 2026-09-18 20:21:25.818439105 +0000 UTC m=+65.693369154 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls") pod "dns-default-6gc77" (UID: "34fb603c-9f1a-46be-9e79-6f6a4f48527b") : secret "dns-default-metrics-tls" not found
Sep 18 20:21:09.821400 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.821382    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:21:09.821682 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:09.821669    2565 scope.go:117] "RemoveContainer" containerID="d6f4a98df1fbdf6655d61385a2137e3879e8d653d5f9933182fa55ba4251127d"
Sep 18 20:21:09.821811 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:09.821794    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-x94rj_openshift-console-operator(6f534f4e-ec58-41c6-a920-68a79947bdbc)\"" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" podUID="6f534f4e-ec58-41c6-a920-68a79947bdbc"
Sep 18 20:21:10.454334 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:10.454307    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkq9j_6fce8f58-7f3c-402c-bd7f-0318ccf013d3/dns-node-resolver/0.log"
Sep 18 20:21:10.525248 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:10.525195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:10.525409 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:10.525276    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:10.525409 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:10.525340    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:10.525504 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:10.525427    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls podName:a45d5039-6364-4256-bb67-b8f2fd3b336e nodeName:}" failed. No retries permitted until 2026-09-18 20:21:12.5254075 +0000 UTC m=+52.400337553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pd9m9" (UID: "a45d5039-6364-4256-bb67-b8f2fd3b336e") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:10.525803 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:10.525785    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a45d5039-6364-4256-bb67-b8f2fd3b336e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:11.331486 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:11.331448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:11.331884 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:11.331610    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:21:11.331884 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:11.331682    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls podName:df51014c-f4e0-460c-84a0-007bc6268194 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:19.331665975 +0000 UTC m=+59.206596022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-78q94" (UID: "df51014c-f4e0-460c-84a0-007bc6268194") : secret "samples-operator-tls" not found
Sep 18 20:21:11.659280 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:11.659184    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8fkmr_f23a7662-58e8-41a3-80e3-bcf87e30af10/node-ca/0.log"
Sep 18 20:21:12.456411 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:12.456384    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c8cpg_d42f20c5-56ec-43f3-9d04-588f1608092e/serve-healthcheck-canary/0.log"
Sep 18 20:21:12.540985 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:12.540956    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:12.541135 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:12.541115    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:12.541204 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:12.541194    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls podName:a45d5039-6364-4256-bb67-b8f2fd3b336e nodeName:}" failed. No retries permitted until 2026-09-18 20:21:16.541173123 +0000 UTC m=+56.416103170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pd9m9" (UID: "a45d5039-6364-4256-bb67-b8f2fd3b336e") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:13.891564 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:13.891529    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:13.892046 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:13.891587    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:13.892046 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:13.892002    2565 scope.go:117] "RemoveContainer" containerID="d6f4a98df1fbdf6655d61385a2137e3879e8d653d5f9933182fa55ba4251127d"
Sep 18 20:21:13.892249 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:13.892187    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-x94rj_openshift-console-operator(6f534f4e-ec58-41c6-a920-68a79947bdbc)\"" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" podUID="6f534f4e-ec58-41c6-a920-68a79947bdbc"
Sep 18 20:21:16.573105 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:16.573062    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:16.573666 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:16.573238    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:16.573666 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:16.573323    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls podName:a45d5039-6364-4256-bb67-b8f2fd3b336e nodeName:}" failed. No retries permitted until 2026-09-18 20:21:24.573304473 +0000 UTC m=+64.448234522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pd9m9" (UID: "a45d5039-6364-4256-bb67-b8f2fd3b336e") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:21:19.392799 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:19.392766    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:19.395023 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:19.395003    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df51014c-f4e0-460c-84a0-007bc6268194-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-78q94\" (UID: \"df51014c-f4e0-460c-84a0-007bc6268194\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:19.466895 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:19.466862    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"
Sep 18 20:21:19.762598 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:19.762569    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgfrj"
Sep 18 20:21:19.770186 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:19.770152    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94"]
Sep 18 20:21:19.846784 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:19.846754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94" event={"ID":"df51014c-f4e0-460c-84a0-007bc6268194","Type":"ContainerStarted","Data":"1416cacc05ca067e565590d5f012f4cf390a668128b07d839ddb0be7dc46ada3"}
Sep 18 20:21:21.853471 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:21.853437    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94" event={"ID":"df51014c-f4e0-460c-84a0-007bc6268194","Type":"ContainerStarted","Data":"cb787a257bd18e635e4d321e29f7248cfeb5dd351f5db4865c31b48521de9e73"}
Sep 18 20:21:21.853861 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:21.853490    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94" event={"ID":"df51014c-f4e0-460c-84a0-007bc6268194","Type":"ContainerStarted","Data":"3992abd684c803f90125a52cc74c015bcbb8f52335c7c571195d845b595928a3"}
Sep 18 20:21:21.881891 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:21.881842    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-78q94" podStartSLOduration=17.277709101 podStartE2EDuration="18.881828149s" totalImagesPullingTime="1.604119048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:21.881071287 +0000 UTC m=+61.756001352" watchObservedRunningTime="2026-09-18 20:21:21.881828149 +0000 UTC m=+61.756758221"
Sep 18 20:21:24.637602 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:24.637567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:24.639793 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:24.639764    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a45d5039-6364-4256-bb67-b8f2fd3b336e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pd9m9\" (UID: \"a45d5039-6364-4256-bb67-b8f2fd3b336e\") " pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:24.643977 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:24.643956    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-299f5\""
Sep 18 20:21:24.652563 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:24.652539    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pd9m9"
Sep 18 20:21:24.923877 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:24.923793    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pd9m9"]
Sep 18 20:21:24.927573 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:24.927546    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45d5039_6364_4256_bb67_b8f2fd3b336e.slice/crio-eeecaf7ebda649421dee3c6d9691c492d76318f07d28b1377881a420461bc87f WatchSource:0}: Error finding container eeecaf7ebda649421dee3c6d9691c492d76318f07d28b1377881a420461bc87f: Status 404 returned error can't find the container with id eeecaf7ebda649421dee3c6d9691c492d76318f07d28b1377881a420461bc87f
Sep 18 20:21:25.849425 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:25.849387    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:21:25.849786 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:25.849438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:21:25.851674 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:25.851646    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34fb603c-9f1a-46be-9e79-6f6a4f48527b-metrics-tls\") pod \"dns-default-6gc77\" (UID: \"34fb603c-9f1a-46be-9e79-6f6a4f48527b\") " pod="openshift-dns/dns-default-6gc77"
Sep 18 20:21:25.851788 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:25.851699    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"image-registry-5bb7fbb4df-xt2cv\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") " pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:21:25.865718 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:25.865689    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pd9m9" event={"ID":"a45d5039-6364-4256-bb67-b8f2fd3b336e","Type":"ContainerStarted","Data":"cb502a53daf25bafb42887013d0e395b7fe1d22daf444756f628f25e97faa37b"}
Sep 18 20:21:25.865833 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:25.865751    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pd9m9" event={"ID":"a45d5039-6364-4256-bb67-b8f2fd3b336e","Type":"ContainerStarted","Data":"eeecaf7ebda649421dee3c6d9691c492d76318f07d28b1377881a420461bc87f"}
Sep 18 20:21:26.132296 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.132204    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pr778\""
Sep 18 20:21:26.138868 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.138846    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:21:26.142202 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.142185    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmtgt\""
Sep 18 20:21:26.149594 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.149576    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gc77"
Sep 18 20:21:26.353954 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.353920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:21:26.356287 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.356270    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:21:26.366403 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.366380    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c08e560-dab1-4bf9-b1a4-beec6c664021-metrics-certs\") pod \"network-metrics-daemon-l56h5\" (UID: \"1c08e560-dab1-4bf9-b1a4-beec6c664021\") " pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:21:26.437024 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.436995    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"]
Sep 18 20:21:26.440433 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:26.440408    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23831892_77d6_4bcf_842f_074248bc8db6.slice/crio-fc76fce4987fae8431225f4e0b993823e780a7a2ad070493598f0efbdd42282c WatchSource:0}: Error finding container fc76fce4987fae8431225f4e0b993823e780a7a2ad070493598f0efbdd42282c: Status 404 returned error can't find the container with id fc76fce4987fae8431225f4e0b993823e780a7a2ad070493598f0efbdd42282c
Sep 18 20:21:26.454431 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.454409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:21:26.456572 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.456554    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:21:26.462198 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.462167    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6gc77"]
Sep 18 20:21:26.465729 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:26.465705    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34fb603c_9f1a_46be_9e79_6f6a4f48527b.slice/crio-0d0b287dc003f6b5ebad44dfab740f73ba5cb2ccd1e1f45f48b33ae7589aff2a WatchSource:0}: Error finding container 0d0b287dc003f6b5ebad44dfab740f73ba5cb2ccd1e1f45f48b33ae7589aff2a: Status 404 returned error can't find the container with id 0d0b287dc003f6b5ebad44dfab740f73ba5cb2ccd1e1f45f48b33ae7589aff2a
Sep 18 20:21:26.467055 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.467036    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0d1d26c8-baa0-4bc3-97bf-ed62049a59f5-original-pull-secret\") pod \"global-pull-secret-syncer-qp8n2\" (UID: \"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5\") " pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:21:26.506907 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.506888    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qp8n2"
Sep 18 20:21:26.522647 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.522622    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27fhx\""
Sep 18 20:21:26.531241 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.531204    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l56h5"
Sep 18 20:21:26.555053 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.555025    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:21:26.557743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.557577    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 20:21:26.568249 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.568174    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 20:21:26.584064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.582272    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z7q2\" (UniqueName: \"kubernetes.io/projected/34b1b8ba-0fdc-433a-afab-64580eb8c2b2-kube-api-access-7z7q2\") pod \"network-check-target-9rqxn\" (UID: \"34b1b8ba-0fdc-433a-afab-64580eb8c2b2\") " pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:21:26.803646 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.803571    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qp8n2"]
Sep 18 20:21:26.807266 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:26.807233    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1d26c8_baa0_4bc3_97bf_ed62049a59f5.slice/crio-82838191cb8d568f26024906fbac0d3664a0d9d98381a4e382103cabbe6d1406 WatchSource:0}: Error finding container 82838191cb8d568f26024906fbac0d3664a0d9d98381a4e382103cabbe6d1406: Status 404 returned error can't find the container with id 82838191cb8d568f26024906fbac0d3664a0d9d98381a4e382103cabbe6d1406
Sep 18 20:21:26.815977 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.815916    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmjzr\""
Sep 18 20:21:26.823928 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.823902    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:21:26.840592 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.840556    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l56h5"]
Sep 18 20:21:26.851854 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:26.851823    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c08e560_dab1_4bf9_b1a4_beec6c664021.slice/crio-8f87b8277b55bb0efccbbf8565589b29f69ee7bf02eb1b42431feffc3cf40aec WatchSource:0}: Error finding container 8f87b8277b55bb0efccbbf8565589b29f69ee7bf02eb1b42431feffc3cf40aec: Status 404 returned error can't find the container with id 8f87b8277b55bb0efccbbf8565589b29f69ee7bf02eb1b42431feffc3cf40aec
Sep 18 20:21:26.874571 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.874497    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qp8n2" event={"ID":"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5","Type":"ContainerStarted","Data":"82838191cb8d568f26024906fbac0d3664a0d9d98381a4e382103cabbe6d1406"}
Sep 18 20:21:26.878955 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.878925    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gc77" event={"ID":"34fb603c-9f1a-46be-9e79-6f6a4f48527b","Type":"ContainerStarted","Data":"0d0b287dc003f6b5ebad44dfab740f73ba5cb2ccd1e1f45f48b33ae7589aff2a"}
Sep 18 20:21:26.882080 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.882037    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pd9m9" event={"ID":"a45d5039-6364-4256-bb67-b8f2fd3b336e","Type":"ContainerStarted","Data":"ca959fa54d4f91cfa3d3ecff2171243ad1c703836becf24922217f97c8999014"}
Sep 18 20:21:26.884897 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.884780    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv" event={"ID":"23831892-77d6-4bcf-842f-074248bc8db6","Type":"ContainerStarted","Data":"5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9"}
Sep 18 20:21:26.884897 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.884831    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv" event={"ID":"23831892-77d6-4bcf-842f-074248bc8db6","Type":"ContainerStarted","Data":"fc76fce4987fae8431225f4e0b993823e780a7a2ad070493598f0efbdd42282c"}
Sep 18 20:21:26.885069 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.884898    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:21:26.886194 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.886164    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l56h5" event={"ID":"1c08e560-dab1-4bf9-b1a4-beec6c664021","Type":"ContainerStarted","Data":"8f87b8277b55bb0efccbbf8565589b29f69ee7bf02eb1b42431feffc3cf40aec"}
Sep 18 20:21:26.904737 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:26.904636    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv" podStartSLOduration=65.904614611 podStartE2EDuration="1m5.904614611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:26.903797692 +0000 UTC m=+66.778727779" watchObservedRunningTime="2026-09-18 20:21:26.904614611 +0000 UTC m=+66.779544676"
Sep 18 20:21:27.122913 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:27.122804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9rqxn"]
Sep 18 20:21:27.595081 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:27.595048    2565 scope.go:117] "RemoveContainer" containerID="d6f4a98df1fbdf6655d61385a2137e3879e8d653d5f9933182fa55ba4251127d"
Sep 18 20:21:28.204125 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:28.204091    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b1b8ba_0fdc_433a_afab_64580eb8c2b2.slice/crio-2b8018a12e3aa5983781cd60cf442ffe7d3f7de50a7cc3c7fdbf7817c429d0b0 WatchSource:0}: Error finding container 2b8018a12e3aa5983781cd60cf442ffe7d3f7de50a7cc3c7fdbf7817c429d0b0: Status 404 returned error can't find the container with id 2b8018a12e3aa5983781cd60cf442ffe7d3f7de50a7cc3c7fdbf7817c429d0b0
Sep 18 20:21:28.899296 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.899176    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l56h5" event={"ID":"1c08e560-dab1-4bf9-b1a4-beec6c664021","Type":"ContainerStarted","Data":"89e0291087c2103eba22fe8ce3410cc3b4077ca20a3717b3e4de7f9703677588"}
Sep 18 20:21:28.899296 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.899259    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l56h5" event={"ID":"1c08e560-dab1-4bf9-b1a4-beec6c664021","Type":"ContainerStarted","Data":"b2ea1e431db8b512455e04d357e5e017e4fd4ee02aac87e9aef034f186ddd168"}
Sep 18 20:21:28.901734 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.901664    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:21:28.901873 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.901751    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" event={"ID":"6f534f4e-ec58-41c6-a920-68a79947bdbc","Type":"ContainerStarted","Data":"b20563d41bc6427180d9000246b75ec85d9b156fb346d19be9f86de91e2bd495"}
Sep 18 20:21:28.902297 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.902246    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:28.905792 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.905766    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gc77" event={"ID":"34fb603c-9f1a-46be-9e79-6f6a4f48527b","Type":"ContainerStarted","Data":"b4122c75079d933de9734c3d842af0c6adae2f75219e4f8c43b5c5b6ea6e1287"}
Sep 18 20:21:28.905896 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.905817    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gc77" event={"ID":"34fb603c-9f1a-46be-9e79-6f6a4f48527b","Type":"ContainerStarted","Data":"9dff896013b020a0fee0de784dec378422f9371d10b5d3d5c890e079e9bb7910"}
Sep 18 20:21:28.906196 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.906176    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6gc77"
Sep 18 20:21:28.907533 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.907510    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9rqxn" event={"ID":"34b1b8ba-0fdc-433a-afab-64580eb8c2b2","Type":"ContainerStarted","Data":"2b8018a12e3aa5983781cd60cf442ffe7d3f7de50a7cc3c7fdbf7817c429d0b0"}
Sep 18 20:21:28.911410 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.911262    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pd9m9" event={"ID":"a45d5039-6364-4256-bb67-b8f2fd3b336e","Type":"ContainerStarted","Data":"5014cd35d4cc29e6dfc7713251f68e69330e1807802dea523edba59739348062"}
Sep 18 20:21:28.915307 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.915263    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l56h5" podStartSLOduration=67.186184425 podStartE2EDuration="1m8.915246987s" totalImagesPullingTime="1.729062562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:28.915196998 +0000 UTC m=+68.790127064" watchObservedRunningTime="2026-09-18 20:21:28.915246987 +0000 UTC m=+68.790177051"
Sep 18 20:21:28.934158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.933982    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pd9m9" podStartSLOduration=17.520663464 podStartE2EDuration="20.933967896s" totalImagesPullingTime="3.413304432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:21:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:28.932646601 +0000 UTC m=+68.807576698" watchObservedRunningTime="2026-09-18 20:21:28.933967896 +0000 UTC m=+68.808897959"
Sep 18 20:21:28.950387 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.950333    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6gc77" podStartSLOduration=32.892258384 podStartE2EDuration="34.950304292s" totalImagesPullingTime="2.058045908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:28.949352949 +0000 UTC m=+68.824283034" watchObservedRunningTime="2026-09-18 20:21:28.950304292 +0000 UTC m=+68.825234357"
Sep 18 20:21:28.967232 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:28.967168    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj" podStartSLOduration=23.111993957 podStartE2EDuration="25.967150349s" totalImagesPullingTime="2.855156392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:28.965960861 +0000 UTC m=+68.840890954" watchObservedRunningTime="2026-09-18 20:21:28.967150349 +0000 UTC m=+68.842080414"
Sep 18 20:21:29.847496 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:29.847457    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-x94rj"
Sep 18 20:21:32.928553 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:32.928510    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qp8n2" event={"ID":"0d1d26c8-baa0-4bc3-97bf-ed62049a59f5","Type":"ContainerStarted","Data":"206953b805ced82012a63cc3a1ac5c04354ac08d00845a73e4dada0b8b03ab4e"}
Sep 18 20:21:32.930174 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:32.930148    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9rqxn" event={"ID":"34b1b8ba-0fdc-433a-afab-64580eb8c2b2","Type":"ContainerStarted","Data":"4c4af34b2868538307b3b34031caa7015b4848e6da07bd3e6021727c5129e4cd"}
Sep 18 20:21:32.930295 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:32.930246    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:21:32.950043 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:32.949997    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qp8n2" podStartSLOduration=66.014620438 podStartE2EDuration="1m11.949981325s" totalImagesPullingTime="5.935360887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:32.949328368 +0000 UTC m=+72.824258432" watchObservedRunningTime="2026-09-18 20:21:32.949981325 +0000 UTC m=+72.824911392"
Sep 18 20:21:32.970735 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:32.970679    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9rqxn" podStartSLOduration=67.504365053 podStartE2EDuration="1m11.970662596s" totalImagesPullingTime="4.466297543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:32.970567522 +0000 UTC m=+72.845497700" watchObservedRunningTime="2026-09-18 20:21:32.970662596 +0000 UTC m=+72.845592662"
Sep 18 20:21:34.728483 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.728447    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"]
Sep 18 20:21:34.731846 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.731822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.733881 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.733858    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 18 20:21:34.734783 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.734766    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 18 20:21:34.735068 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.735016    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 20:21:34.735349 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.735324    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 20:21:34.736262 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.736243    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 20:21:34.736468 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.736454    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 18 20:21:34.736521 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.736487    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 18 20:21:34.764495 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.764467    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"]
Sep 18 20:21:34.821049 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.821014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.821237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.821063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/719b9e06-5219-41bc-8cd8-7c202fa4562f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.821237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.821083    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.821237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.821160    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-hub\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.821237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.821217    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w46bw\" (UniqueName: \"kubernetes.io/projected/719b9e06-5219-41bc-8cd8-7c202fa4562f-kube-api-access-w46bw\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.821376 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.821245    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-ca\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.921667 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.921637    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/719b9e06-5219-41bc-8cd8-7c202fa4562f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.921667 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.921671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.921874 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.921696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-hub\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.921874 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.921723    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w46bw\" (UniqueName: \"kubernetes.io/projected/719b9e06-5219-41bc-8cd8-7c202fa4562f-kube-api-access-w46bw\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.921874 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.921781    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-ca\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.921874 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.921836    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.922501 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.922474    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/719b9e06-5219-41bc-8cd8-7c202fa4562f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.924293 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.924272    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.924422 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.924310    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-ca\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.924476 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.924458    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-hub\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.924516 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.924456    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/719b9e06-5219-41bc-8cd8-7c202fa4562f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:34.933215 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:34.933180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w46bw\" (UniqueName: \"kubernetes.io/projected/719b9e06-5219-41bc-8cd8-7c202fa4562f-kube-api-access-w46bw\") pod \"cluster-proxy-proxy-agent-7c958567b8-9dx4b\" (UID: \"719b9e06-5219-41bc-8cd8-7c202fa4562f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:35.057628 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:35.057546    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"
Sep 18 20:21:35.348670 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:35.348600    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b"]
Sep 18 20:21:35.351654 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:35.351631    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod719b9e06_5219_41bc_8cd8_7c202fa4562f.slice/crio-459df3cd7eeac0826d6cb171068c34fa2e80ab61a3f181c59aadf401aa9dac02 WatchSource:0}: Error finding container 459df3cd7eeac0826d6cb171068c34fa2e80ab61a3f181c59aadf401aa9dac02: Status 404 returned error can't find the container with id 459df3cd7eeac0826d6cb171068c34fa2e80ab61a3f181c59aadf401aa9dac02
Sep 18 20:21:35.942891 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:35.942850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b" event={"ID":"719b9e06-5219-41bc-8cd8-7c202fa4562f","Type":"ContainerStarted","Data":"459df3cd7eeac0826d6cb171068c34fa2e80ab61a3f181c59aadf401aa9dac02"}
Sep 18 20:21:38.918889 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:38.918808    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6gc77"
Sep 18 20:21:38.957984 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:38.957951    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b" event={"ID":"719b9e06-5219-41bc-8cd8-7c202fa4562f","Type":"ContainerStarted","Data":"100936a584d28bb38107c4bf2ec0377cb89dbf0b4d43b8f46c023fad25bb3dee"}
Sep 18 20:21:38.958158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:38.957996    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b" event={"ID":"719b9e06-5219-41bc-8cd8-7c202fa4562f","Type":"ContainerStarted","Data":"ee2824f104ebd90f777a9f102e289f503435c7da35a454ef05e421630fe0727f"}
Sep 18 20:21:38.958158 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:38.958009    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b" event={"ID":"719b9e06-5219-41bc-8cd8-7c202fa4562f","Type":"ContainerStarted","Data":"a04a741d2ab5ea2425c102118d16689315e9eebc5d06e8c31300ec1c35645d28"}
Sep 18 20:21:38.979549 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:38.979492    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c958567b8-9dx4b" podStartSLOduration=1.698706578 podStartE2EDuration="4.979469412s" totalImagesPullingTime="3.280762834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:38.978497865 +0000 UTC m=+78.853427923" watchObservedRunningTime="2026-09-18 20:21:38.979469412 +0000 UTC m=+78.854399477"
Sep 18 20:21:47.895858 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:47.895729    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:21:48.291725 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.291688    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g4rjg"]
Sep 18 20:21:48.294883 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.294864    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.296908 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.296889    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 20:21:48.297010 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.296992    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 20:21:48.297106 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.297088    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 20:21:48.297432 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.297400    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 20:21:48.297495 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.297417    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 20:21:48.297726 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.297713    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-42k9n\""
Sep 18 20:21:48.300720 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.300698    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 20:21:48.418929 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.418899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnjk8\" (UniqueName: \"kubernetes.io/projected/3148d0bb-8388-4795-b985-e653b00e2b59-kube-api-access-lnjk8\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419096 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.418937    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-textfile\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419096 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.418967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-tls\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419096 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.419036    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-wtmp\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419096 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.419065    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3148d0bb-8388-4795-b985-e653b00e2b59-metrics-client-ca\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419096 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.419089    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-sys\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419268 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.419102    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-root\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419268 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.419135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.419268 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.419192    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520459 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-tls\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520459 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-wtmp\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520477    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3148d0bb-8388-4795-b985-e653b00e2b59-metrics-client-ca\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520494    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-sys\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-root\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520588    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-sys\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520612    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-root\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:48.520597    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520620    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-wtmp\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.520694 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:21:48.520694    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-tls podName:3148d0bb-8388-4795-b985-e653b00e2b59 nodeName:}" failed. No retries permitted until 2026-09-18 20:21:49.020672626 +0000 UTC m=+88.895602682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-tls") pod "node-exporter-g4rjg" (UID: "3148d0bb-8388-4795-b985-e653b00e2b59") : secret "node-exporter-tls" not found
Sep 18 20:21:48.521122 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnjk8\" (UniqueName: \"kubernetes.io/projected/3148d0bb-8388-4795-b985-e653b00e2b59-kube-api-access-lnjk8\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.521122 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.520825    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-textfile\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.521122 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.521107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-textfile\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.521228 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.521158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3148d0bb-8388-4795-b985-e653b00e2b59-metrics-client-ca\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.521228 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.521165    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.522906 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.522885    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:48.537144 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:48.537117    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnjk8\" (UniqueName: \"kubernetes.io/projected/3148d0bb-8388-4795-b985-e653b00e2b59-kube-api-access-lnjk8\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:49.024306 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:49.024266    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-tls\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:49.026510 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:49.026490    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3148d0bb-8388-4795-b985-e653b00e2b59-node-exporter-tls\") pod \"node-exporter-g4rjg\" (UID: \"3148d0bb-8388-4795-b985-e653b00e2b59\") " pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:49.204228 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:49.204184    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g4rjg"
Sep 18 20:21:49.215879 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:21:49.215851    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3148d0bb_8388_4795_b985_e653b00e2b59.slice/crio-148de5398550c381db4c772ec047dbd2116740a7935cb44982cc18eedea7dca1 WatchSource:0}: Error finding container 148de5398550c381db4c772ec047dbd2116740a7935cb44982cc18eedea7dca1: Status 404 returned error can't find the container with id 148de5398550c381db4c772ec047dbd2116740a7935cb44982cc18eedea7dca1
Sep 18 20:21:49.996865 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:49.996822    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4rjg" event={"ID":"3148d0bb-8388-4795-b985-e653b00e2b59","Type":"ContainerStarted","Data":"148de5398550c381db4c772ec047dbd2116740a7935cb44982cc18eedea7dca1"}
Sep 18 20:21:51.001902 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:51.001867    2565 generic.go:356] "Generic (PLEG): container finished" podID="3148d0bb-8388-4795-b985-e653b00e2b59" containerID="3545c9d8c1847ba6ef11e5f57bf00c371d9c1f61baed91609ea19dd8a7e9e503" exitCode=0
Sep 18 20:21:51.002317 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:51.001919    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4rjg" event={"ID":"3148d0bb-8388-4795-b985-e653b00e2b59","Type":"ContainerDied","Data":"3545c9d8c1847ba6ef11e5f57bf00c371d9c1f61baed91609ea19dd8a7e9e503"}
Sep 18 20:21:52.007455 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:52.007419    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4rjg" event={"ID":"3148d0bb-8388-4795-b985-e653b00e2b59","Type":"ContainerStarted","Data":"36ce806c45f54dce4a0b52665dd9f39ceadb6c8b27b99ce7461b93ac8f9bd238"}
Sep 18 20:21:52.007833 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:52.007502    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4rjg" event={"ID":"3148d0bb-8388-4795-b985-e653b00e2b59","Type":"ContainerStarted","Data":"263dc8117677a66c521d5ef9838a1d7bcfa2ce1bffdf77f209850218b785963a"}
Sep 18 20:21:52.055969 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:52.055917    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g4rjg" podStartSLOduration=3.147280393 podStartE2EDuration="4.055902742s" totalImagesPullingTime="847.20096ms" totalInitContainerRuntime="61.421389ms" isStatefulPod=true podCreationTimestamp="2026-09-18 20:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:52.055331534 +0000 UTC m=+91.930261616" watchObservedRunningTime="2026-09-18 20:21:52.055902742 +0000 UTC m=+91.930832831"
Sep 18 20:21:58.269711 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:21:58.269674    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"]
Sep 18 20:22:03.936449 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:03.936416    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9rqxn"
Sep 18 20:22:23.109890 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.109855    2565 generic.go:356] "Generic (PLEG): container finished" podID="4e36a61f-4a55-4981-be3c-b23fce9b617e" containerID="307e1a061d016c34aa589fd4b9b32ddd22c71a4ee1ce8663aad8be29ad017b8f" exitCode=0
Sep 18 20:22:23.110384 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.109932    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll" event={"ID":"4e36a61f-4a55-4981-be3c-b23fce9b617e","Type":"ContainerDied","Data":"307e1a061d016c34aa589fd4b9b32ddd22c71a4ee1ce8663aad8be29ad017b8f"}
Sep 18 20:22:23.110384 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.110334    2565 scope.go:117] "RemoveContainer" containerID="307e1a061d016c34aa589fd4b9b32ddd22c71a4ee1ce8663aad8be29ad017b8f"
Sep 18 20:22:23.289514 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.289476    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv" podUID="23831892-77d6-4bcf-842f-074248bc8db6" containerName="registry" containerID="cri-o://5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9" gracePeriod=30
Sep 18 20:22:23.532145 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.532120    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:22:23.668831 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668746    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23831892-77d6-4bcf-842f-074248bc8db6-ca-trust-extracted\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.668831 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668782    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-installation-pull-secrets\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.668831 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668808    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-registry-certificates\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.669110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668837    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.669110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668867    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-bound-sa-token\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.669110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668888    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-image-registry-private-configuration\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.669110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668923    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-trusted-ca\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.669110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.668972    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfgdw\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-kube-api-access-rfgdw\") pod \"23831892-77d6-4bcf-842f-074248bc8db6\" (UID: \"23831892-77d6-4bcf-842f-074248bc8db6\") "
Sep 18 20:22:23.669411 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.669181    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:23.669593 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.669551    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:23.671796 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.671752    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-kube-api-access-rfgdw" (OuterVolumeSpecName: "kube-api-access-rfgdw") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "kube-api-access-rfgdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:23.671967 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.671934    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:23.672033 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.671992    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:23.672082 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.672036    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:23.672082 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.672030    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:23.677737 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.677713    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23831892-77d6-4bcf-842f-074248bc8db6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "23831892-77d6-4bcf-842f-074248bc8db6" (UID: "23831892-77d6-4bcf-842f-074248bc8db6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:23.769954 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.769924    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-trusted-ca\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:23.769954 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.769950    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfgdw\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-kube-api-access-rfgdw\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:23.769954 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.769962    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23831892-77d6-4bcf-842f-074248bc8db6-ca-trust-extracted\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:23.770152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.769973    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-installation-pull-secrets\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:23.770152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.769983    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23831892-77d6-4bcf-842f-074248bc8db6-registry-certificates\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:23.770152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.769992    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-registry-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:23.770152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.770000    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23831892-77d6-4bcf-842f-074248bc8db6-bound-sa-token\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:23.770152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:23.770008    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23831892-77d6-4bcf-842f-074248bc8db6-image-registry-private-configuration\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:22:24.115431 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.115396    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6q2ll" event={"ID":"4e36a61f-4a55-4981-be3c-b23fce9b617e","Type":"ContainerStarted","Data":"e040ff6b6d7d3664812a6115cab3ddbf5dbbcbac49abf65798f50c901499d15e"}
Sep 18 20:22:24.116583 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.116560    2565 generic.go:356] "Generic (PLEG): container finished" podID="23831892-77d6-4bcf-842f-074248bc8db6" containerID="5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9" exitCode=0
Sep 18 20:22:24.116695 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.116605    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"
Sep 18 20:22:24.116695 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.116607    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv" event={"ID":"23831892-77d6-4bcf-842f-074248bc8db6","Type":"ContainerDied","Data":"5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9"}
Sep 18 20:22:24.116805 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.116732    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bb7fbb4df-xt2cv" event={"ID":"23831892-77d6-4bcf-842f-074248bc8db6","Type":"ContainerDied","Data":"fc76fce4987fae8431225f4e0b993823e780a7a2ad070493598f0efbdd42282c"}
Sep 18 20:22:24.116805 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.116758    2565 scope.go:117] "RemoveContainer" containerID="5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9"
Sep 18 20:22:24.125427 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.125408    2565 scope.go:117] "RemoveContainer" containerID="5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9"
Sep 18 20:22:24.125712 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:22:24.125685    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9\": container with ID starting with 5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9 not found: ID does not exist" containerID="5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9"
Sep 18 20:22:24.125763 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.125723    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9"} err="failed to get container status \"5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9\": rpc error: code = NotFound desc = could not find container \"5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9\": container with ID starting with 5a0406478962ecab5934e7494e33e6cb1ae9daa422560cd16e1fb73d4c151ff9 not found: ID does not exist"
Sep 18 20:22:24.148787 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.148758    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"]
Sep 18 20:22:24.150104 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.150081    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bb7fbb4df-xt2cv"]
Sep 18 20:22:24.599548 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:22:24.599516    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23831892-77d6-4bcf-842f-074248bc8db6" path="/var/lib/kubelet/pods/23831892-77d6-4bcf-842f-074248bc8db6/volumes"
Sep 18 20:25:09.003732 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.003692    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"]
Sep 18 20:25:09.004149 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.003920    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23831892-77d6-4bcf-842f-074248bc8db6" containerName="registry"
Sep 18 20:25:09.004149 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.003931    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="23831892-77d6-4bcf-842f-074248bc8db6" containerName="registry"
Sep 18 20:25:09.004149 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.003978    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="23831892-77d6-4bcf-842f-074248bc8db6" containerName="registry"
Sep 18 20:25:09.006778 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.006759    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.009051 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.009030    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 20:25:09.009165 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.009030    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 20:25:09.009531 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.009516    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hrpl6\""
Sep 18 20:25:09.016061 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.016040    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"]
Sep 18 20:25:09.077849 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.077814    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.078011 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.077852    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.078011 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.077882    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6llq\" (UniqueName: \"kubernetes.io/projected/a67102c9-9aed-47ac-b76f-9ff0921efe99-kube-api-access-f6llq\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.178724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.178694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.178841 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.178732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.178841 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.178752    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6llq\" (UniqueName: \"kubernetes.io/projected/a67102c9-9aed-47ac-b76f-9ff0921efe99-kube-api-access-f6llq\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.179093 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.179071    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.179170 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.179138    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.187821 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.187795    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6llq\" (UniqueName: \"kubernetes.io/projected/a67102c9-9aed-47ac-b76f-9ff0921efe99-kube-api-access-f6llq\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.315884 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.315786    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:09.615388 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.615316    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"]
Sep 18 20:25:09.626000 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:25:09.625974    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67102c9_9aed_47ac_b76f_9ff0921efe99.slice/crio-7068f61ff8b2db16294108055349fd7aef90a94929d05b997fb4939a2e9c3dc0 WatchSource:0}: Error finding container 7068f61ff8b2db16294108055349fd7aef90a94929d05b997fb4939a2e9c3dc0: Status 404 returned error can't find the container with id 7068f61ff8b2db16294108055349fd7aef90a94929d05b997fb4939a2e9c3dc0
Sep 18 20:25:09.647335 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:09.647303    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf" event={"ID":"a67102c9-9aed-47ac-b76f-9ff0921efe99","Type":"ContainerStarted","Data":"7068f61ff8b2db16294108055349fd7aef90a94929d05b997fb4939a2e9c3dc0"}
Sep 18 20:25:16.675670 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:16.675633    2565 generic.go:356] "Generic (PLEG): container finished" podID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerID="0aded4330aaa8730e3186424ac8ea88d1dff687d17fe62b104bfc4ae13df88bf" exitCode=0
Sep 18 20:25:16.675670 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:16.675675    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf" event={"ID":"a67102c9-9aed-47ac-b76f-9ff0921efe99","Type":"ContainerDied","Data":"0aded4330aaa8730e3186424ac8ea88d1dff687d17fe62b104bfc4ae13df88bf"}
Sep 18 20:25:19.691287 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:19.691243    2565 generic.go:356] "Generic (PLEG): container finished" podID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerID="40d66fd3aac27686be4461ff5cb03c3972cad7bae7a91a4ab1e7cec25ff6abf0" exitCode=0
Sep 18 20:25:19.691765 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:19.691348    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf" event={"ID":"a67102c9-9aed-47ac-b76f-9ff0921efe99","Type":"ContainerDied","Data":"40d66fd3aac27686be4461ff5cb03c3972cad7bae7a91a4ab1e7cec25ff6abf0"}
Sep 18 20:25:20.541245 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:20.541199    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:25:20.545727 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:20.545699    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:25:20.556837 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:20.556815    2565 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 20:25:26.717535 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:26.717502    2565 generic.go:356] "Generic (PLEG): container finished" podID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerID="03b984d455bb955616896c2cab121079badf908b0ac5f31c83c7fb0e534378b2" exitCode=0
Sep 18 20:25:26.717535 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:26.717537    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf" event={"ID":"a67102c9-9aed-47ac-b76f-9ff0921efe99","Type":"ContainerDied","Data":"03b984d455bb955616896c2cab121079badf908b0ac5f31c83c7fb0e534378b2"}
Sep 18 20:25:27.840962 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:27.840935    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:27.922499 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:27.922467    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-bundle\") pod \"a67102c9-9aed-47ac-b76f-9ff0921efe99\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") "
Sep 18 20:25:27.922668 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:27.922539    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6llq\" (UniqueName: \"kubernetes.io/projected/a67102c9-9aed-47ac-b76f-9ff0921efe99-kube-api-access-f6llq\") pod \"a67102c9-9aed-47ac-b76f-9ff0921efe99\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") "
Sep 18 20:25:27.922668 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:27.922561    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-util\") pod \"a67102c9-9aed-47ac-b76f-9ff0921efe99\" (UID: \"a67102c9-9aed-47ac-b76f-9ff0921efe99\") "
Sep 18 20:25:27.923247 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:27.923202    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-bundle" (OuterVolumeSpecName: "bundle") pod "a67102c9-9aed-47ac-b76f-9ff0921efe99" (UID: "a67102c9-9aed-47ac-b76f-9ff0921efe99"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:25:27.924700 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:27.924680    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a67102c9-9aed-47ac-b76f-9ff0921efe99-kube-api-access-f6llq" (OuterVolumeSpecName: "kube-api-access-f6llq") pod "a67102c9-9aed-47ac-b76f-9ff0921efe99" (UID: "a67102c9-9aed-47ac-b76f-9ff0921efe99"). InnerVolumeSpecName "kube-api-access-f6llq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:25:27.928460 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:27.928435    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-util" (OuterVolumeSpecName: "util") pod "a67102c9-9aed-47ac-b76f-9ff0921efe99" (UID: "a67102c9-9aed-47ac-b76f-9ff0921efe99"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:25:28.023169 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:28.023093    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6llq\" (UniqueName: \"kubernetes.io/projected/a67102c9-9aed-47ac-b76f-9ff0921efe99-kube-api-access-f6llq\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:25:28.023169 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:28.023119    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-util\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:25:28.023169 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:28.023129    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a67102c9-9aed-47ac-b76f-9ff0921efe99-bundle\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:25:28.725274 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:28.725243    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf" event={"ID":"a67102c9-9aed-47ac-b76f-9ff0921efe99","Type":"ContainerDied","Data":"7068f61ff8b2db16294108055349fd7aef90a94929d05b997fb4939a2e9c3dc0"}
Sep 18 20:25:28.725454 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:28.725298    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7068f61ff8b2db16294108055349fd7aef90a94929d05b997fb4939a2e9c3dc0"
Sep 18 20:25:28.725454 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:28.725258    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2nsmf"
Sep 18 20:25:32.316744 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.316707    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"]
Sep 18 20:25:32.317124 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.316935    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerName="extract"
Sep 18 20:25:32.317124 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.316945    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerName="extract"
Sep 18 20:25:32.317124 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.316957    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerName="pull"
Sep 18 20:25:32.317124 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.316964    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerName="pull"
Sep 18 20:25:32.317124 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.316976    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerName="util"
Sep 18 20:25:32.317124 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.316982    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerName="util"
Sep 18 20:25:32.317124 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.317022    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a67102c9-9aed-47ac-b76f-9ff0921efe99" containerName="extract"
Sep 18 20:25:32.356547 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.356519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"]
Sep 18 20:25:32.356703 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.356637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:32.358960 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.358934    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-7nw56\""
Sep 18 20:25:32.361449 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.361431    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 18 20:25:32.361613 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.361591    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 18 20:25:32.361794 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.361769    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 18 20:25:32.453426 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.453389    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq6lv\" (UniqueName: \"kubernetes.io/projected/13021781-ca82-4e25-a513-0f6cf4bcd361-kube-api-access-hq6lv\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-jbdz7\" (UID: \"13021781-ca82-4e25-a513-0f6cf4bcd361\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:32.453589 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.453450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/13021781-ca82-4e25-a513-0f6cf4bcd361-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-jbdz7\" (UID: \"13021781-ca82-4e25-a513-0f6cf4bcd361\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:32.554458 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.554424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/13021781-ca82-4e25-a513-0f6cf4bcd361-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-jbdz7\" (UID: \"13021781-ca82-4e25-a513-0f6cf4bcd361\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:32.554546 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.554468    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq6lv\" (UniqueName: \"kubernetes.io/projected/13021781-ca82-4e25-a513-0f6cf4bcd361-kube-api-access-hq6lv\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-jbdz7\" (UID: \"13021781-ca82-4e25-a513-0f6cf4bcd361\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:32.556790 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.556758    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/13021781-ca82-4e25-a513-0f6cf4bcd361-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-jbdz7\" (UID: \"13021781-ca82-4e25-a513-0f6cf4bcd361\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:32.562904 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.562882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq6lv\" (UniqueName: \"kubernetes.io/projected/13021781-ca82-4e25-a513-0f6cf4bcd361-kube-api-access-hq6lv\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-jbdz7\" (UID: \"13021781-ca82-4e25-a513-0f6cf4bcd361\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:32.666804 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:32.666724    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:33.026030 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:33.025995    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"]
Sep 18 20:25:33.053020 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:25:33.052994    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13021781_ca82_4e25_a513_0f6cf4bcd361.slice/crio-03fb6ba81f0e1c3e3c4efa302667f0b5f60bbc6a8efd7b40911097ce7a69c56d WatchSource:0}: Error finding container 03fb6ba81f0e1c3e3c4efa302667f0b5f60bbc6a8efd7b40911097ce7a69c56d: Status 404 returned error can't find the container with id 03fb6ba81f0e1c3e3c4efa302667f0b5f60bbc6a8efd7b40911097ce7a69c56d
Sep 18 20:25:33.054833 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:33.054815    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:25:33.745281 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:33.745237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7" event={"ID":"13021781-ca82-4e25-a513-0f6cf4bcd361","Type":"ContainerStarted","Data":"03fb6ba81f0e1c3e3c4efa302667f0b5f60bbc6a8efd7b40911097ce7a69c56d"}
Sep 18 20:25:36.694371 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.694336    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-9dzh9"]
Sep 18 20:25:36.696607 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.696591    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.698754 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.698724    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 18 20:25:36.698885 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.698863    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-pv8jd\""
Sep 18 20:25:36.698972 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.698885    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 18 20:25:36.709491 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.709467    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-9dzh9"]
Sep 18 20:25:36.757699 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.757669    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7" event={"ID":"13021781-ca82-4e25-a513-0f6cf4bcd361","Type":"ContainerStarted","Data":"aa8a9d69726869b924ff0d4a599364cc12465c1a3a336bb5f0a9ce9854ab35b2"}
Sep 18 20:25:36.757842 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.757777    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:25:36.781147 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.781100    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7" podStartSLOduration=1.863033478 podStartE2EDuration="4.781085926s" totalImagesPullingTime="2.918052448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:25:36.779238383 +0000 UTC m=+316.654168465" watchObservedRunningTime="2026-09-18 20:25:36.781085926 +0000 UTC m=+316.656015990"
Sep 18 20:25:36.786543 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.786519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.786662 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.786555    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndrrn\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-kube-api-access-ndrrn\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.786662 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.786625    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/fb4a9688-9e4c-40cd-b8f3-73faa8071538-cabundle0\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.887909 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.887881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/fb4a9688-9e4c-40cd-b8f3-73faa8071538-cabundle0\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.887909 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.887912    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.888128 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.887935    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndrrn\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-kube-api-access-ndrrn\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.888128 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:36.888047    2565 secret.go:281] references non-existent secret key: ca.crt
Sep 18 20:25:36.888128 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:36.888067    2565 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 18 20:25:36.888128 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:36.888079    2565 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-9dzh9: references non-existent secret key: ca.crt
Sep 18 20:25:36.888346 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:36.888151    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates podName:fb4a9688-9e4c-40cd-b8f3-73faa8071538 nodeName:}" failed. No retries permitted until 2026-09-18 20:25:37.388132734 +0000 UTC m=+317.263062787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates") pod "keda-operator-579c8958f7-9dzh9" (UID: "fb4a9688-9e4c-40cd-b8f3-73faa8071538") : references non-existent secret key: ca.crt
Sep 18 20:25:36.888752 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.888727    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/fb4a9688-9e4c-40cd-b8f3-73faa8071538-cabundle0\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:36.896980 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:36.896955    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndrrn\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-kube-api-access-ndrrn\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:37.338592 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.338535    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"]
Sep 18 20:25:37.341055 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.341039    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.342982 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.342959    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 18 20:25:37.350720 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.350699    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"]
Sep 18 20:25:37.392019 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.391990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:37.392131 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:37.392107    2565 secret.go:281] references non-existent secret key: ca.crt
Sep 18 20:25:37.392131 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:37.392119    2565 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 18 20:25:37.392131 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:37.392126    2565 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-9dzh9: references non-existent secret key: ca.crt
Sep 18 20:25:37.392273 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:25:37.392178    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates podName:fb4a9688-9e4c-40cd-b8f3-73faa8071538 nodeName:}" failed. No retries permitted until 2026-09-18 20:25:38.392165486 +0000 UTC m=+318.267095529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates") pod "keda-operator-579c8958f7-9dzh9" (UID: "fb4a9688-9e4c-40cd-b8f3-73faa8071538") : references non-existent secret key: ca.crt
Sep 18 20:25:37.493289 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.493252    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/194c1595-cba2-4a6e-b746-84e3e0f696ac-certificates\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.493460 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.493325    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6qnk\" (UniqueName: \"kubernetes.io/projected/194c1595-cba2-4a6e-b746-84e3e0f696ac-kube-api-access-x6qnk\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.493460 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.493381    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/194c1595-cba2-4a6e-b746-84e3e0f696ac-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.594392 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.594319    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6qnk\" (UniqueName: \"kubernetes.io/projected/194c1595-cba2-4a6e-b746-84e3e0f696ac-kube-api-access-x6qnk\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.594392 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.594362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/194c1595-cba2-4a6e-b746-84e3e0f696ac-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.594575 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.594395    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/194c1595-cba2-4a6e-b746-84e3e0f696ac-certificates\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.594840 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.594814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/194c1595-cba2-4a6e-b746-84e3e0f696ac-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.596856 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.596839    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/194c1595-cba2-4a6e-b746-84e3e0f696ac-certificates\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.609966 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.609940    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6qnk\" (UniqueName: \"kubernetes.io/projected/194c1595-cba2-4a6e-b746-84e3e0f696ac-kube-api-access-x6qnk\") pod \"keda-metrics-apiserver-77857b7b98-7cl92\" (UID: \"194c1595-cba2-4a6e-b746-84e3e0f696ac\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:37.651565 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:37.651542    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:38.369268 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:38.369235    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"]
Sep 18 20:25:38.401201 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:38.401173    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:38.403554 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:38.403530    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fb4a9688-9e4c-40cd-b8f3-73faa8071538-certificates\") pod \"keda-operator-579c8958f7-9dzh9\" (UID: \"fb4a9688-9e4c-40cd-b8f3-73faa8071538\") " pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:38.506402 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:38.506366    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:38.632891 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:25:38.632863    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod194c1595_cba2_4a6e_b746_84e3e0f696ac.slice/crio-24ee0f31c89a56fc480ba8d77fdd7e88a047bdf4d86cce26abf00d49e5b531e7 WatchSource:0}: Error finding container 24ee0f31c89a56fc480ba8d77fdd7e88a047bdf4d86cce26abf00d49e5b531e7: Status 404 returned error can't find the container with id 24ee0f31c89a56fc480ba8d77fdd7e88a047bdf4d86cce26abf00d49e5b531e7
Sep 18 20:25:38.765989 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:38.765961    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92" event={"ID":"194c1595-cba2-4a6e-b746-84e3e0f696ac","Type":"ContainerStarted","Data":"24ee0f31c89a56fc480ba8d77fdd7e88a047bdf4d86cce26abf00d49e5b531e7"}
Sep 18 20:25:39.020761 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:39.020725    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-9dzh9"]
Sep 18 20:25:39.749837 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:25:39.749806    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4a9688_9e4c_40cd_b8f3_73faa8071538.slice/crio-37f612c7f5a3c34dbdfa77f1501ae2bc01ec84d6d188c9c35937f4ca5079e0a4 WatchSource:0}: Error finding container 37f612c7f5a3c34dbdfa77f1501ae2bc01ec84d6d188c9c35937f4ca5079e0a4: Status 404 returned error can't find the container with id 37f612c7f5a3c34dbdfa77f1501ae2bc01ec84d6d188c9c35937f4ca5079e0a4
Sep 18 20:25:39.771624 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:39.771585    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-9dzh9" event={"ID":"fb4a9688-9e4c-40cd-b8f3-73faa8071538","Type":"ContainerStarted","Data":"37f612c7f5a3c34dbdfa77f1501ae2bc01ec84d6d188c9c35937f4ca5079e0a4"}
Sep 18 20:25:41.782460 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:41.782401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92" event={"ID":"194c1595-cba2-4a6e-b746-84e3e0f696ac","Type":"ContainerStarted","Data":"70b1c9738a46bcebff9c3ed2bd1a5687418d9ff1d7cc2f1c35c1e94eaace6371"}
Sep 18 20:25:41.783137 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:41.782619    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:41.800793 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:41.800738    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92" podStartSLOduration=2.626464228 podStartE2EDuration="4.800711236s" totalImagesPullingTime="2.174247008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:25:41.799125255 +0000 UTC m=+321.674055341" watchObservedRunningTime="2026-09-18 20:25:41.800711236 +0000 UTC m=+321.675641302"
Sep 18 20:25:43.792236 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:43.792136    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-9dzh9" event={"ID":"fb4a9688-9e4c-40cd-b8f3-73faa8071538","Type":"ContainerStarted","Data":"34b870710f0c4174c8eab313a1afc5d920416bfc8b6e3776a4e9af0870142e55"}
Sep 18 20:25:43.792615 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:43.792283    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:25:43.808667 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:43.808595    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-9dzh9" podStartSLOduration=4.077704863 podStartE2EDuration="7.808575881s" totalImagesPullingTime="3.730871018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:25:43.807790957 +0000 UTC m=+323.682721039" watchObservedRunningTime="2026-09-18 20:25:43.808575881 +0000 UTC m=+323.683505946"
Sep 18 20:25:52.791626 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:52.791599    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-7cl92"
Sep 18 20:25:57.764424 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:25:57.764382    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-jbdz7"
Sep 18 20:26:04.799050 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:26:04.799020    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-9dzh9"
Sep 18 20:27:02.818854 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.818818    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"]
Sep 18 20:27:02.821993 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.821968    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:02.824164 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.824143    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 18 20:27:02.824290 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.824173    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-d4wrd\""
Sep 18 20:27:02.824736 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.824721    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 18 20:27:02.824808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.824766    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 18 20:27:02.842840 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.842815    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"]
Sep 18 20:27:02.870570 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.870545    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-xgxz5"]
Sep 18 20:27:02.873431 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.873415    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:02.875617 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.875601    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-7klnw\""
Sep 18 20:27:02.875857 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.875841    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 18 20:27:02.894193 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.894168    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-xgxz5"]
Sep 18 20:27:02.908186 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.908165    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9h7c\" (UniqueName: \"kubernetes.io/projected/24ea8626-3cab-408b-a734-53697f31c346-kube-api-access-d9h7c\") pod \"llmisvc-controller-manager-645f9c8fd6-rv5wv\" (UID: \"24ea8626-3cab-408b-a734-53697f31c346\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:02.908301 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:02.908224    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24ea8626-3cab-408b-a734-53697f31c346-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-rv5wv\" (UID: \"24ea8626-3cab-408b-a734-53697f31c346\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:03.009334 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.009298    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24ea8626-3cab-408b-a734-53697f31c346-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-rv5wv\" (UID: \"24ea8626-3cab-408b-a734-53697f31c346\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:03.009532 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.009351    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8khh\" (UniqueName: \"kubernetes.io/projected/624f42b8-68d4-4789-b8b6-b4d7d4f89613-kube-api-access-g8khh\") pod \"seaweedfs-64568bcd49-xgxz5\" (UID: \"624f42b8-68d4-4789-b8b6-b4d7d4f89613\") " pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:03.009532 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.009416    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h7c\" (UniqueName: \"kubernetes.io/projected/24ea8626-3cab-408b-a734-53697f31c346-kube-api-access-d9h7c\") pod \"llmisvc-controller-manager-645f9c8fd6-rv5wv\" (UID: \"24ea8626-3cab-408b-a734-53697f31c346\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:03.009532 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.009448    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/624f42b8-68d4-4789-b8b6-b4d7d4f89613-data\") pod \"seaweedfs-64568bcd49-xgxz5\" (UID: \"624f42b8-68d4-4789-b8b6-b4d7d4f89613\") " pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:03.009532 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:27:03.009461    2565 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 18 20:27:03.009677 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:27:03.009539    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ea8626-3cab-408b-a734-53697f31c346-cert podName:24ea8626-3cab-408b-a734-53697f31c346 nodeName:}" failed. No retries permitted until 2026-09-18 20:27:03.509519826 +0000 UTC m=+403.384449875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24ea8626-3cab-408b-a734-53697f31c346-cert") pod "llmisvc-controller-manager-645f9c8fd6-rv5wv" (UID: "24ea8626-3cab-408b-a734-53697f31c346") : secret "llmisvc-webhook-server-cert" not found
Sep 18 20:27:03.020909 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.020880    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9h7c\" (UniqueName: \"kubernetes.io/projected/24ea8626-3cab-408b-a734-53697f31c346-kube-api-access-d9h7c\") pod \"llmisvc-controller-manager-645f9c8fd6-rv5wv\" (UID: \"24ea8626-3cab-408b-a734-53697f31c346\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:03.110257 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.110153    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/624f42b8-68d4-4789-b8b6-b4d7d4f89613-data\") pod \"seaweedfs-64568bcd49-xgxz5\" (UID: \"624f42b8-68d4-4789-b8b6-b4d7d4f89613\") " pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:03.110257 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.110226    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8khh\" (UniqueName: \"kubernetes.io/projected/624f42b8-68d4-4789-b8b6-b4d7d4f89613-kube-api-access-g8khh\") pod \"seaweedfs-64568bcd49-xgxz5\" (UID: \"624f42b8-68d4-4789-b8b6-b4d7d4f89613\") " pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:03.110541 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.110521    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/624f42b8-68d4-4789-b8b6-b4d7d4f89613-data\") pod \"seaweedfs-64568bcd49-xgxz5\" (UID: \"624f42b8-68d4-4789-b8b6-b4d7d4f89613\") " pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:03.119311 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.119281    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8khh\" (UniqueName: \"kubernetes.io/projected/624f42b8-68d4-4789-b8b6-b4d7d4f89613-kube-api-access-g8khh\") pod \"seaweedfs-64568bcd49-xgxz5\" (UID: \"624f42b8-68d4-4789-b8b6-b4d7d4f89613\") " pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:03.182509 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.182460    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:03.452415 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.452384    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-xgxz5"]
Sep 18 20:27:03.458535 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:27:03.458505    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624f42b8_68d4_4789_b8b6_b4d7d4f89613.slice/crio-d264403439a5b3f5cc231cb90e8f5e01ad9c713e816df2cfc7b42ffbf442e56d WatchSource:0}: Error finding container d264403439a5b3f5cc231cb90e8f5e01ad9c713e816df2cfc7b42ffbf442e56d: Status 404 returned error can't find the container with id d264403439a5b3f5cc231cb90e8f5e01ad9c713e816df2cfc7b42ffbf442e56d
Sep 18 20:27:03.513083 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.513052    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24ea8626-3cab-408b-a734-53697f31c346-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-rv5wv\" (UID: \"24ea8626-3cab-408b-a734-53697f31c346\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:03.515358 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.515338    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24ea8626-3cab-408b-a734-53697f31c346-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-rv5wv\" (UID: \"24ea8626-3cab-408b-a734-53697f31c346\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:03.731536 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:03.731452    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:04.005129 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:04.005043    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"]
Sep 18 20:27:04.009219 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:27:04.009177    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod24ea8626_3cab_408b_a734_53697f31c346.slice/crio-b0f86c9f2050116f94742b3818687e085ce48a7ea3d178858c5de6a976c0227c WatchSource:0}: Error finding container b0f86c9f2050116f94742b3818687e085ce48a7ea3d178858c5de6a976c0227c: Status 404 returned error can't find the container with id b0f86c9f2050116f94742b3818687e085ce48a7ea3d178858c5de6a976c0227c
Sep 18 20:27:04.081434 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:04.081390    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv" event={"ID":"24ea8626-3cab-408b-a734-53697f31c346","Type":"ContainerStarted","Data":"b0f86c9f2050116f94742b3818687e085ce48a7ea3d178858c5de6a976c0227c"}
Sep 18 20:27:04.082810 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:04.082772    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-xgxz5" event={"ID":"624f42b8-68d4-4789-b8b6-b4d7d4f89613","Type":"ContainerStarted","Data":"d264403439a5b3f5cc231cb90e8f5e01ad9c713e816df2cfc7b42ffbf442e56d"}
Sep 18 20:27:08.100796 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:08.100754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv" event={"ID":"24ea8626-3cab-408b-a734-53697f31c346","Type":"ContainerStarted","Data":"d4fca86ee7e1c2704fc27259a98a3c87db0344ece043f88176ed2fe88f7740fa"}
Sep 18 20:27:08.101302 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:08.100845    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:27:08.102521 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:08.102497    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-xgxz5" event={"ID":"624f42b8-68d4-4789-b8b6-b4d7d4f89613","Type":"ContainerStarted","Data":"683368f4402fb7b7311ae44153cde6e59d8166c40ecec6989fe3aa57795e6053"}
Sep 18 20:27:08.102634 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:08.102544    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:08.118956 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:08.118915    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv" podStartSLOduration=2.687907423 podStartE2EDuration="6.118898656s" totalImagesPullingTime="3.430991233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:27:08.117360094 +0000 UTC m=+407.992290158" watchObservedRunningTime="2026-09-18 20:27:08.118898656 +0000 UTC m=+407.993828720"
Sep 18 20:27:08.136511 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:08.136470    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-xgxz5" podStartSLOduration=2.102698253 podStartE2EDuration="6.136457992s" totalImagesPullingTime="4.033759739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:27:08.134657663 +0000 UTC m=+408.009587748" watchObservedRunningTime="2026-09-18 20:27:08.136457992 +0000 UTC m=+408.011388056"
Sep 18 20:27:14.109180 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:14.109147    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-xgxz5"
Sep 18 20:27:39.108373 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:27:39.108342    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-rv5wv"
Sep 18 20:28:22.162349 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.162309    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-l64jg"]
Sep 18 20:28:22.166563 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.166523    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.169721 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.169698    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-cpr8b\""
Sep 18 20:28:22.169849 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.169731    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 18 20:28:22.177619 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.177593    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-l64jg"]
Sep 18 20:28:22.188277 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.188254    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-lsbkj"]
Sep 18 20:28:22.191394 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.191375    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.195052 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.195031    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-bm6dn\""
Sep 18 20:28:22.195282 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.195267    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 18 20:28:22.195282 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.195274    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 18 20:28:22.201903 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.201880    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-lsbkj"]
Sep 18 20:28:22.302542 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.302511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af1be1a5-f375-48bb-88ea-284cdf494981-tls-certs\") pod \"model-serving-api-7b954ffc95-l64jg\" (UID: \"af1be1a5-f375-48bb-88ea-284cdf494981\") " pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.302729 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.302552    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk5gh\" (UniqueName: \"kubernetes.io/projected/65afc7a3-249e-46d5-bf36-eb9adb0e7510-kube-api-access-nk5gh\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.302729 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.302589    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-cert\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.302729 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.302615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-metrics-certs\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.302729 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.302631    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb295\" (UniqueName: \"kubernetes.io/projected/af1be1a5-f375-48bb-88ea-284cdf494981-kube-api-access-sb295\") pod \"model-serving-api-7b954ffc95-l64jg\" (UID: \"af1be1a5-f375-48bb-88ea-284cdf494981\") " pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.402956 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.402927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-cert\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.403121 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.402959    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-metrics-certs\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.403121 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.403001    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb295\" (UniqueName: \"kubernetes.io/projected/af1be1a5-f375-48bb-88ea-284cdf494981-kube-api-access-sb295\") pod \"model-serving-api-7b954ffc95-l64jg\" (UID: \"af1be1a5-f375-48bb-88ea-284cdf494981\") " pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.403121 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.403063    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af1be1a5-f375-48bb-88ea-284cdf494981-tls-certs\") pod \"model-serving-api-7b954ffc95-l64jg\" (UID: \"af1be1a5-f375-48bb-88ea-284cdf494981\") " pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.403121 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:28:22.403091    2565 secret.go:189] Couldn't get secret kserve/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found
Sep 18 20:28:22.403358 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.403099    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk5gh\" (UniqueName: \"kubernetes.io/projected/65afc7a3-249e-46d5-bf36-eb9adb0e7510-kube-api-access-nk5gh\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.403358 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:28:22.403174    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-metrics-certs podName:65afc7a3-249e-46d5-bf36-eb9adb0e7510 nodeName:}" failed. No retries permitted until 2026-09-18 20:28:22.903154353 +0000 UTC m=+482.778084395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-metrics-certs") pod "odh-model-controller-7b966d7984-lsbkj" (UID: "65afc7a3-249e-46d5-bf36-eb9adb0e7510") : secret "odh-model-controller-metrics-tls" not found
Sep 18 20:28:22.405471 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.405450    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-cert\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.405589 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.405565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af1be1a5-f375-48bb-88ea-284cdf494981-tls-certs\") pod \"model-serving-api-7b954ffc95-l64jg\" (UID: \"af1be1a5-f375-48bb-88ea-284cdf494981\") " pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.413092 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.413043    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb295\" (UniqueName: \"kubernetes.io/projected/af1be1a5-f375-48bb-88ea-284cdf494981-kube-api-access-sb295\") pod \"model-serving-api-7b954ffc95-l64jg\" (UID: \"af1be1a5-f375-48bb-88ea-284cdf494981\") " pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.413201 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.413084    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk5gh\" (UniqueName: \"kubernetes.io/projected/65afc7a3-249e-46d5-bf36-eb9adb0e7510-kube-api-access-nk5gh\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.477821 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.477790    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:22.753035 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.753002    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-l64jg"]
Sep 18 20:28:22.756024 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:28:22.756001    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1be1a5_f375_48bb_88ea_284cdf494981.slice/crio-9857cbd38b638896a0630520639c98044056bd16242151b111c725ad10f513ab WatchSource:0}: Error finding container 9857cbd38b638896a0630520639c98044056bd16242151b111c725ad10f513ab: Status 404 returned error can't find the container with id 9857cbd38b638896a0630520639c98044056bd16242151b111c725ad10f513ab
Sep 18 20:28:22.907981 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.907946    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-metrics-certs\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:22.910293 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:22.910271    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65afc7a3-249e-46d5-bf36-eb9adb0e7510-metrics-certs\") pod \"odh-model-controller-7b966d7984-lsbkj\" (UID: \"65afc7a3-249e-46d5-bf36-eb9adb0e7510\") " pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:23.100680 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:23.100597    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:23.375702 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:23.375670    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-lsbkj"]
Sep 18 20:28:23.378502 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:28:23.378474    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65afc7a3_249e_46d5_bf36_eb9adb0e7510.slice/crio-1c2431e102dcadc6bbcc0f53700ee8e666c843d9140dd2cf82664ff13aadadfe WatchSource:0}: Error finding container 1c2431e102dcadc6bbcc0f53700ee8e666c843d9140dd2cf82664ff13aadadfe: Status 404 returned error can't find the container with id 1c2431e102dcadc6bbcc0f53700ee8e666c843d9140dd2cf82664ff13aadadfe
Sep 18 20:28:23.383640 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:23.383614    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-l64jg" event={"ID":"af1be1a5-f375-48bb-88ea-284cdf494981","Type":"ContainerStarted","Data":"9857cbd38b638896a0630520639c98044056bd16242151b111c725ad10f513ab"}
Sep 18 20:28:24.392629 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:24.392586    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-lsbkj" event={"ID":"65afc7a3-249e-46d5-bf36-eb9adb0e7510","Type":"ContainerStarted","Data":"1c2431e102dcadc6bbcc0f53700ee8e666c843d9140dd2cf82664ff13aadadfe"}
Sep 18 20:28:24.395695 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:24.395666    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-l64jg" event={"ID":"af1be1a5-f375-48bb-88ea-284cdf494981","Type":"ContainerStarted","Data":"d25ecc1ac3a079fe586db411d09006e2046f6b5c83f73250f3549f2ca2228668"}
Sep 18 20:28:24.395834 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:24.395802    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:24.414329 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:24.414283    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-l64jg" podStartSLOduration=0.956392232 podStartE2EDuration="2.414263367s" totalImagesPullingTime="1.457871135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:24.41294063 +0000 UTC m=+484.287870693" watchObservedRunningTime="2026-09-18 20:28:24.414263367 +0000 UTC m=+484.289193432"
Sep 18 20:28:26.410184 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:26.410146    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-lsbkj" event={"ID":"65afc7a3-249e-46d5-bf36-eb9adb0e7510","Type":"ContainerStarted","Data":"93a61dd273ea7a2db08d506954c689c3b62830d1b3f1553f3ab37e13bd011787"}
Sep 18 20:28:26.410582 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:26.410243    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:28:26.427369 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:26.427311    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-lsbkj" podStartSLOduration=1.991123409 podStartE2EDuration="4.42729065s" totalImagesPullingTime="2.436167241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:26.424933673 +0000 UTC m=+486.299863756" watchObservedRunningTime="2026-09-18 20:28:26.42729065 +0000 UTC m=+486.302220719"
Sep 18 20:28:35.409304 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:35.409273    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-l64jg"
Sep 18 20:28:37.421601 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:28:37.421569    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-lsbkj"
Sep 18 20:30:09.334264 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.334228    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"]
Sep 18 20:30:09.337457 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.337440    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.339690 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.339668    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ef587-predictor-serving-cert\""
Sep 18 20:30:09.340001 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.339985    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 18 20:30:09.340584 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.340566    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 18 20:30:09.340584 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.340573    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ef587-kube-rbac-proxy-sar-config\""
Sep 18 20:30:09.340710 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.340573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6hbfs\""
Sep 18 20:30:09.348340 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.348318    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"]
Sep 18 20:30:09.416847 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.416810    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.416847 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.416844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lgsw\" (UniqueName: \"kubernetes.io/projected/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-api-access-5lgsw\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.417048 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.416921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.518289 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.518253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.518481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.518314    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.518481 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:30:09.518423    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-ef587-predictor-serving-cert: secret "error-404-isvc-ef587-predictor-serving-cert" not found
Sep 18 20:30:09.518481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.518430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lgsw\" (UniqueName: \"kubernetes.io/projected/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-api-access-5lgsw\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.518643 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:30:09.518505    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls podName:b46ded42-af6c-42f3-b9ec-b165b9a1f26e nodeName:}" failed. No retries permitted until 2026-09-18 20:30:10.018483282 +0000 UTC m=+589.893413339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls") pod "error-404-isvc-ef587-predictor-df4f8586-q6bxt" (UID: "b46ded42-af6c-42f3-b9ec-b165b9a1f26e") : secret "error-404-isvc-ef587-predictor-serving-cert" not found
Sep 18 20:30:09.518991 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.518972    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:09.528973 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:09.528947    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lgsw\" (UniqueName: \"kubernetes.io/projected/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-api-access-5lgsw\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:10.021957 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:10.021919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:10.024668 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:10.024649    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls\") pod \"error-404-isvc-ef587-predictor-df4f8586-q6bxt\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") " pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:10.246771 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:10.246735    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:10.518377 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:10.518337    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"]
Sep 18 20:30:10.522258 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:30:10.522231    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb46ded42_af6c_42f3_b9ec_b165b9a1f26e.slice/crio-8e45af2c33f3a81a08c8c72f88eb29d529d8e9c6eb244f3fbc8e4f1b36456399 WatchSource:0}: Error finding container 8e45af2c33f3a81a08c8c72f88eb29d529d8e9c6eb244f3fbc8e4f1b36456399: Status 404 returned error can't find the container with id 8e45af2c33f3a81a08c8c72f88eb29d529d8e9c6eb244f3fbc8e4f1b36456399
Sep 18 20:30:10.791993 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:10.791959    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" event={"ID":"b46ded42-af6c-42f3-b9ec-b165b9a1f26e","Type":"ContainerStarted","Data":"8e45af2c33f3a81a08c8c72f88eb29d529d8e9c6eb244f3fbc8e4f1b36456399"}
Sep 18 20:30:22.423064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:22.423034    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:30:22.423064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:22.423051    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:30:22.859190 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:22.859155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" event={"ID":"b46ded42-af6c-42f3-b9ec-b165b9a1f26e","Type":"ContainerStarted","Data":"f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197"}
Sep 18 20:30:24.869667 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:24.869634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" event={"ID":"b46ded42-af6c-42f3-b9ec-b165b9a1f26e","Type":"ContainerStarted","Data":"44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7"}
Sep 18 20:30:24.870095 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:24.869838    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:24.870095 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:24.869957    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:24.871132 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:24.871100    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:30:24.888347 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:24.888308    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podStartSLOduration=1.711702264 podStartE2EDuration="15.888294665s" totalImagesPullingTime="14.176592401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:24.887068373 +0000 UTC m=+604.761998462" watchObservedRunningTime="2026-09-18 20:30:24.888294665 +0000 UTC m=+604.763224729"
Sep 18 20:30:25.873994 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:25.873956    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:30:30.878092 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:30.878062    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:30:30.880590 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:30.878663    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:30:40.879100 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:40.879056    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:30:50.879089 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:30:50.879009    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:31:00.879518 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:31:00.879476    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:31:10.878743 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:31:10.878698    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:31:20.879864 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:31:20.879831    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:32:00.191549 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.191511    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"]
Sep 18 20:32:00.192045 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.191793    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" containerID="cri-o://f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197" gracePeriod=30
Sep 18 20:32:00.192045 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.191884    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kube-rbac-proxy" containerID="cri-o://44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7" gracePeriod=30
Sep 18 20:32:00.256016 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.255985    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"]
Sep 18 20:32:00.259349 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.259335    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.261488 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.261466    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5b4a7-predictor-serving-cert\""
Sep 18 20:32:00.261488 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.261469    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5b4a7-kube-rbac-proxy-sar-config\""
Sep 18 20:32:00.272123 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.272102    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"]
Sep 18 20:32:00.393359 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.393322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.393528 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.393396    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b117c588-1478-4cf8-8979-c74972d34bca-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.393528 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.393467    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqbs8\" (UniqueName: \"kubernetes.io/projected/b117c588-1478-4cf8-8979-c74972d34bca-kube-api-access-wqbs8\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.494521 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.494431    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b117c588-1478-4cf8-8979-c74972d34bca-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.494521 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.494469    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqbs8\" (UniqueName: \"kubernetes.io/projected/b117c588-1478-4cf8-8979-c74972d34bca-kube-api-access-wqbs8\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.494521 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.494506    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.494765 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:32:00.494618    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-serving-cert: secret "error-404-isvc-5b4a7-predictor-serving-cert" not found
Sep 18 20:32:00.494765 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:32:00.494692    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls podName:b117c588-1478-4cf8-8979-c74972d34bca nodeName:}" failed. No retries permitted until 2026-09-18 20:32:00.994674961 +0000 UTC m=+700.869605003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls") pod "error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" (UID: "b117c588-1478-4cf8-8979-c74972d34bca") : secret "error-404-isvc-5b4a7-predictor-serving-cert" not found
Sep 18 20:32:00.495140 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.495121    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b117c588-1478-4cf8-8979-c74972d34bca-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.503171 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.503142    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqbs8\" (UniqueName: \"kubernetes.io/projected/b117c588-1478-4cf8-8979-c74972d34bca-kube-api-access-wqbs8\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:00.874323 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.874229    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.22:8643/healthz\": dial tcp 10.132.0.22:8643: connect: connection refused"
Sep 18 20:32:00.879572 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.879548    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 18 20:32:00.998481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:00.998446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:01.000815 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:01.000790    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls\") pod \"error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") " pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:01.169104 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:01.169015    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:01.258090 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:01.258058    2565 generic.go:356] "Generic (PLEG): container finished" podID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerID="44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7" exitCode=2
Sep 18 20:32:01.258437 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:01.258129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" event={"ID":"b46ded42-af6c-42f3-b9ec-b165b9a1f26e","Type":"ContainerDied","Data":"44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7"}
Sep 18 20:32:01.442644 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:01.442619    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"]
Sep 18 20:32:01.444873 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:32:01.444836    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb117c588_1478_4cf8_8979_c74972d34bca.slice/crio-11bbc788d5981f0ae9859ee6baae541bfec340fd07c60f2c666144cfeade8b12 WatchSource:0}: Error finding container 11bbc788d5981f0ae9859ee6baae541bfec340fd07c60f2c666144cfeade8b12: Status 404 returned error can't find the container with id 11bbc788d5981f0ae9859ee6baae541bfec340fd07c60f2c666144cfeade8b12
Sep 18 20:32:02.264133 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:02.264096    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" event={"ID":"b117c588-1478-4cf8-8979-c74972d34bca","Type":"ContainerStarted","Data":"868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846"}
Sep 18 20:32:02.264649 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:02.264149    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" event={"ID":"b117c588-1478-4cf8-8979-c74972d34bca","Type":"ContainerStarted","Data":"dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec"}
Sep 18 20:32:02.264649 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:02.264171    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" event={"ID":"b117c588-1478-4cf8-8979-c74972d34bca","Type":"ContainerStarted","Data":"11bbc788d5981f0ae9859ee6baae541bfec340fd07c60f2c666144cfeade8b12"}
Sep 18 20:32:02.264649 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:02.264261    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:02.284123 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:02.284064    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podStartSLOduration=2.284050507 podStartE2EDuration="2.284050507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:02.283381085 +0000 UTC m=+702.158311173" watchObservedRunningTime="2026-09-18 20:32:02.284050507 +0000 UTC m=+702.158980570"
Sep 18 20:32:03.268105 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.268078    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:03.269552 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.269516    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 18 20:32:03.832552 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.832528    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:32:03.923118 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.923033    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-rbac-proxy-sar-config\") pod \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") "
Sep 18 20:32:03.923118 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.923074    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lgsw\" (UniqueName: \"kubernetes.io/projected/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-api-access-5lgsw\") pod \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") "
Sep 18 20:32:03.923118 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.923094    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls\") pod \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\" (UID: \"b46ded42-af6c-42f3-b9ec-b165b9a1f26e\") "
Sep 18 20:32:03.923441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.923419    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b46ded42-af6c-42f3-b9ec-b165b9a1f26e" (UID: "b46ded42-af6c-42f3-b9ec-b165b9a1f26e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:03.925349 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.925319    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b46ded42-af6c-42f3-b9ec-b165b9a1f26e" (UID: "b46ded42-af6c-42f3-b9ec-b165b9a1f26e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:03.925466 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:03.925353    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-api-access-5lgsw" (OuterVolumeSpecName: "kube-api-access-5lgsw") pod "b46ded42-af6c-42f3-b9ec-b165b9a1f26e" (UID: "b46ded42-af6c-42f3-b9ec-b165b9a1f26e"). InnerVolumeSpecName "kube-api-access-5lgsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:04.023985 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.023951    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:32:04.023985 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.023979    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lgsw\" (UniqueName: \"kubernetes.io/projected/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-kube-api-access-5lgsw\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:32:04.023985 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.023991    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b46ded42-af6c-42f3-b9ec-b165b9a1f26e-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:32:04.273221 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.273176    2565 generic.go:356] "Generic (PLEG): container finished" podID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerID="f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197" exitCode=0
Sep 18 20:32:04.273625 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.273244    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"
Sep 18 20:32:04.273625 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.273243    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" event={"ID":"b46ded42-af6c-42f3-b9ec-b165b9a1f26e","Type":"ContainerDied","Data":"f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197"}
Sep 18 20:32:04.273625 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.273301    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt" event={"ID":"b46ded42-af6c-42f3-b9ec-b165b9a1f26e","Type":"ContainerDied","Data":"8e45af2c33f3a81a08c8c72f88eb29d529d8e9c6eb244f3fbc8e4f1b36456399"}
Sep 18 20:32:04.273625 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.273328    2565 scope.go:117] "RemoveContainer" containerID="44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7"
Sep 18 20:32:04.273914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.273875    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 18 20:32:04.284784 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.284763    2565 scope.go:117] "RemoveContainer" containerID="f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197"
Sep 18 20:32:04.292968 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.292948    2565 scope.go:117] "RemoveContainer" containerID="44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7"
Sep 18 20:32:04.293308 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:32:04.293285    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7\": container with ID starting with 44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7 not found: ID does not exist" containerID="44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7"
Sep 18 20:32:04.293410 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.293313    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7"} err="failed to get container status \"44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7\": rpc error: code = NotFound desc = could not find container \"44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7\": container with ID starting with 44fc760ad904480ff247f7f5ae71c8ba7512cf8b07de090bd9ca9ae3d23c12c7 not found: ID does not exist"
Sep 18 20:32:04.293410 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.293329    2565 scope.go:117] "RemoveContainer" containerID="f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197"
Sep 18 20:32:04.293669 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:32:04.293625    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197\": container with ID starting with f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197 not found: ID does not exist" containerID="f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197"
Sep 18 20:32:04.293767 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.293679    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197"} err="failed to get container status \"f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197\": rpc error: code = NotFound desc = could not find container \"f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197\": container with ID starting with f519e2b4fd5faf542f940c6c0086e2d692c9989cfca25db5744724b18177a197 not found: ID does not exist"
Sep 18 20:32:04.295562 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.295540    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"]
Sep 18 20:32:04.299352 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.299331    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ef587-predictor-df4f8586-q6bxt"]
Sep 18 20:32:04.600693 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:04.600612    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" path="/var/lib/kubelet/pods/b46ded42-af6c-42f3-b9ec-b165b9a1f26e/volumes"
Sep 18 20:32:09.278276 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:09.278247    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:09.278691 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:09.278660    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 18 20:32:19.279534 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:19.279446    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 18 20:32:29.279366 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:29.279323    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 18 20:32:30.894826 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.894796    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"]
Sep 18 20:32:30.895253 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.895081    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kube-rbac-proxy"
Sep 18 20:32:30.895253 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.895092    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kube-rbac-proxy"
Sep 18 20:32:30.895253 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.895113    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container"
Sep 18 20:32:30.895253 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.895119    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container"
Sep 18 20:32:30.895253 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.895163    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kube-rbac-proxy"
Sep 18 20:32:30.895253 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.895172    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b46ded42-af6c-42f3-b9ec-b165b9a1f26e" containerName="kserve-container"
Sep 18 20:32:30.898811 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.898784    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:30.900989 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.900970    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2bced-predictor-serving-cert\""
Sep 18 20:32:30.901095 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.900977    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2bced-kube-rbac-proxy-sar-config\""
Sep 18 20:32:30.906730 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:30.906711    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"]
Sep 18 20:32:31.023221 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.023181    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6c7r\" (UniqueName: \"kubernetes.io/projected/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-api-access-k6c7r\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.023394 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.023287    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef5aba97-3d63-4bd9-8090-df52ac3112f9-proxy-tls\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.023394 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.023345    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.124049 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.124013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6c7r\" (UniqueName: \"kubernetes.io/projected/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-api-access-k6c7r\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.124334 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.124075    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef5aba97-3d63-4bd9-8090-df52ac3112f9-proxy-tls\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.124334 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.124117    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.124784 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.124763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.126458 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.126439    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef5aba97-3d63-4bd9-8090-df52ac3112f9-proxy-tls\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.131684 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.131661    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6c7r\" (UniqueName: \"kubernetes.io/projected/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-api-access-k6c7r\") pod \"error-404-isvc-2bced-predictor-66fb58dcfb-jlz89\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") " pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.209748 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.209712    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:31.477669 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:31.477646    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"]
Sep 18 20:32:31.480191 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:32:31.480164    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5aba97_3d63_4bd9_8090_df52ac3112f9.slice/crio-ab4f9efab663c7c56afd8258b4f08ac5de6a794251878939a09f2440b85bf072 WatchSource:0}: Error finding container ab4f9efab663c7c56afd8258b4f08ac5de6a794251878939a09f2440b85bf072: Status 404 returned error can't find the container with id ab4f9efab663c7c56afd8258b4f08ac5de6a794251878939a09f2440b85bf072
Sep 18 20:32:32.391655 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:32.391617    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" event={"ID":"ef5aba97-3d63-4bd9-8090-df52ac3112f9","Type":"ContainerStarted","Data":"4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b"}
Sep 18 20:32:32.392044 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:32.391684    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" event={"ID":"ef5aba97-3d63-4bd9-8090-df52ac3112f9","Type":"ContainerStarted","Data":"66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6"}
Sep 18 20:32:32.392044 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:32.391716    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" event={"ID":"ef5aba97-3d63-4bd9-8090-df52ac3112f9","Type":"ContainerStarted","Data":"ab4f9efab663c7c56afd8258b4f08ac5de6a794251878939a09f2440b85bf072"}
Sep 18 20:32:32.392044 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:32.391777    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:32.408610 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:32.408549    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podStartSLOduration=2.408529447 podStartE2EDuration="2.408529447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:32.408229879 +0000 UTC m=+732.283159962" watchObservedRunningTime="2026-09-18 20:32:32.408529447 +0000 UTC m=+732.283459504"
Sep 18 20:32:33.396752 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:33.396722    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:33.398488 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:33.398459    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:32:34.406499 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:34.406452    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:32:39.279282 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:39.279244    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 18 20:32:39.412374 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:39.412347    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:32:39.412819 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:39.412783    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:32:49.278759 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:49.278721    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 18 20:32:49.413297 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:49.413256    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:32:59.279387 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:59.279355    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:32:59.412894 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:32:59.412852    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:33:09.413426 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:33:09.413389    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:33:19.412894 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:33:19.412854    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:33:29.414065 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:33:29.414035    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:35:22.453908 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:35:22.453878    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:35:22.454401 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:35:22.454326    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:40:22.478953 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:40:22.478922    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:40:22.480781 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:40:22.480760    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:41:25.619917 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.619839    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"]
Sep 18 20:41:25.622456 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.620074    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container" containerID="cri-o://dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec" gracePeriod=30
Sep 18 20:41:25.622456 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.620127    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kube-rbac-proxy" containerID="cri-o://868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846" gracePeriod=30
Sep 18 20:41:25.698557 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.698521    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"]
Sep 18 20:41:25.701810 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.701795    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.703949 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.703928    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c092f-predictor-serving-cert\""
Sep 18 20:41:25.704059 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.703932    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c092f-kube-rbac-proxy-sar-config\""
Sep 18 20:41:25.712322 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.712301    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"]
Sep 18 20:41:25.755264 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.755224    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.755402 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.755275    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.755402 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.755312    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js8w6\" (UniqueName: \"kubernetes.io/projected/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-api-access-js8w6\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.855825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.855788    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.855825 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.855833    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.856050 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.855867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js8w6\" (UniqueName: \"kubernetes.io/projected/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-api-access-js8w6\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.856050 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:41:25.855971    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-c092f-predictor-serving-cert: secret "error-404-isvc-c092f-predictor-serving-cert" not found
Sep 18 20:41:25.856131 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:41:25.856050    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls podName:5738ee2e-80d0-48aa-b8fd-11f37f76c1d3 nodeName:}" failed. No retries permitted until 2026-09-18 20:41:26.356029511 +0000 UTC m=+1266.230959574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls") pod "error-404-isvc-c092f-predictor-64b7d65596-7k47n" (UID: "5738ee2e-80d0-48aa-b8fd-11f37f76c1d3") : secret "error-404-isvc-c092f-predictor-serving-cert" not found
Sep 18 20:41:25.856563 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.856544    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:25.864118 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:25.864099    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js8w6\" (UniqueName: \"kubernetes.io/projected/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-api-access-js8w6\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:26.359774 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:26.359731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:26.362154 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:26.362123    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls\") pod \"error-404-isvc-c092f-predictor-64b7d65596-7k47n\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") " pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:26.571575 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:26.571542    2565 generic.go:356] "Generic (PLEG): container finished" podID="b117c588-1478-4cf8-8979-c74972d34bca" containerID="868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846" exitCode=2
Sep 18 20:41:26.571575 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:26.571581    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" event={"ID":"b117c588-1478-4cf8-8979-c74972d34bca","Type":"ContainerDied","Data":"868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846"}
Sep 18 20:41:26.612771 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:26.612692    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:26.890115 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:26.890020    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"]
Sep 18 20:41:26.894016 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:41:26.893978    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5738ee2e_80d0_48aa_b8fd_11f37f76c1d3.slice/crio-9da3b40fd0015bf29925f277eaa68cde5acbfe5a6e9071f59bd6ff888acef721 WatchSource:0}: Error finding container 9da3b40fd0015bf29925f277eaa68cde5acbfe5a6e9071f59bd6ff888acef721: Status 404 returned error can't find the container with id 9da3b40fd0015bf29925f277eaa68cde5acbfe5a6e9071f59bd6ff888acef721
Sep 18 20:41:27.577938 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:27.577900    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" event={"ID":"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3","Type":"ContainerStarted","Data":"8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9"}
Sep 18 20:41:27.578111 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:27.577959    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" event={"ID":"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3","Type":"ContainerStarted","Data":"6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f"}
Sep 18 20:41:27.578111 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:27.577981    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" event={"ID":"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3","Type":"ContainerStarted","Data":"9da3b40fd0015bf29925f277eaa68cde5acbfe5a6e9071f59bd6ff888acef721"}
Sep 18 20:41:27.578187 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:27.578128    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:27.599434 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:27.599360    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podStartSLOduration=2.599344973 podStartE2EDuration="2.599344973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:41:27.598571577 +0000 UTC m=+1267.473501659" watchObservedRunningTime="2026-09-18 20:41:27.599344973 +0000 UTC m=+1267.474275037"
Sep 18 20:41:28.581785 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:28.581755    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:28.583143 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:28.583114    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 18 20:41:29.056810 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.056787    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:41:29.082294 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.082265    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b117c588-1478-4cf8-8979-c74972d34bca-kube-rbac-proxy-sar-config\") pod \"b117c588-1478-4cf8-8979-c74972d34bca\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") "
Sep 18 20:41:29.082445 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.082332    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls\") pod \"b117c588-1478-4cf8-8979-c74972d34bca\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") "
Sep 18 20:41:29.082445 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.082352    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqbs8\" (UniqueName: \"kubernetes.io/projected/b117c588-1478-4cf8-8979-c74972d34bca-kube-api-access-wqbs8\") pod \"b117c588-1478-4cf8-8979-c74972d34bca\" (UID: \"b117c588-1478-4cf8-8979-c74972d34bca\") "
Sep 18 20:41:29.082690 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.082665    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b117c588-1478-4cf8-8979-c74972d34bca-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b117c588-1478-4cf8-8979-c74972d34bca" (UID: "b117c588-1478-4cf8-8979-c74972d34bca"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:41:29.084407 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.084385    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b117c588-1478-4cf8-8979-c74972d34bca" (UID: "b117c588-1478-4cf8-8979-c74972d34bca"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:41:29.084485 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.084448    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b117c588-1478-4cf8-8979-c74972d34bca-kube-api-access-wqbs8" (OuterVolumeSpecName: "kube-api-access-wqbs8") pod "b117c588-1478-4cf8-8979-c74972d34bca" (UID: "b117c588-1478-4cf8-8979-c74972d34bca"). InnerVolumeSpecName "kube-api-access-wqbs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:41:29.183579 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.183545    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b117c588-1478-4cf8-8979-c74972d34bca-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:41:29.183579 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.183578    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqbs8\" (UniqueName: \"kubernetes.io/projected/b117c588-1478-4cf8-8979-c74972d34bca-kube-api-access-wqbs8\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:41:29.183770 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.183590    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b117c588-1478-4cf8-8979-c74972d34bca-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:41:29.587055 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.586965    2565 generic.go:356] "Generic (PLEG): container finished" podID="b117c588-1478-4cf8-8979-c74972d34bca" containerID="dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec" exitCode=0
Sep 18 20:41:29.587055 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.587021    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"
Sep 18 20:41:29.587055 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.587040    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" event={"ID":"b117c588-1478-4cf8-8979-c74972d34bca","Type":"ContainerDied","Data":"dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec"}
Sep 18 20:41:29.587569 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.587099    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8" event={"ID":"b117c588-1478-4cf8-8979-c74972d34bca","Type":"ContainerDied","Data":"11bbc788d5981f0ae9859ee6baae541bfec340fd07c60f2c666144cfeade8b12"}
Sep 18 20:41:29.587569 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.587125    2565 scope.go:117] "RemoveContainer" containerID="868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846"
Sep 18 20:41:29.587712 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.587686    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 18 20:41:29.596269 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.596128    2565 scope.go:117] "RemoveContainer" containerID="dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec"
Sep 18 20:41:29.604223 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.604194    2565 scope.go:117] "RemoveContainer" containerID="868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846"
Sep 18 20:41:29.604571 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:41:29.604550    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846\": container with ID starting with 868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846 not found: ID does not exist" containerID="868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846"
Sep 18 20:41:29.604676 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.604579    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846"} err="failed to get container status \"868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846\": rpc error: code = NotFound desc = could not find container \"868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846\": container with ID starting with 868f2110192e5997d45152537454d82f154a8fd0eda1b1793ce0a3f1fa623846 not found: ID does not exist"
Sep 18 20:41:29.604676 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.604600    2565 scope.go:117] "RemoveContainer" containerID="dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec"
Sep 18 20:41:29.604896 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:41:29.604877    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec\": container with ID starting with dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec not found: ID does not exist" containerID="dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec"
Sep 18 20:41:29.604983 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.604903    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec"} err="failed to get container status \"dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec\": rpc error: code = NotFound desc = could not find container \"dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec\": container with ID starting with dadfb2b1843adf9e2652d38fd5413dfd7033840124c671f6246e82e1e0e960ec not found: ID does not exist"
Sep 18 20:41:29.608463 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.608440    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"]
Sep 18 20:41:29.611374 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:29.611354    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b4a7-predictor-7d4878b6fd-489v8"]
Sep 18 20:41:30.599858 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:30.599827    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b117c588-1478-4cf8-8979-c74972d34bca" path="/var/lib/kubelet/pods/b117c588-1478-4cf8-8979-c74972d34bca/volumes"
Sep 18 20:41:34.592324 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:34.592289    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:41:34.592808 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:34.592779    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 18 20:41:44.593496 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:44.593451    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 18 20:41:54.593458 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:54.593417    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 18 20:41:56.345105 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.345067    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"]
Sep 18 20:41:56.345605 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.345293    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" containerID="cri-o://66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6" gracePeriod=30
Sep 18 20:41:56.345605 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.345358    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kube-rbac-proxy" containerID="cri-o://4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b" gracePeriod=30
Sep 18 20:41:56.512951 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.512911    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"]
Sep 18 20:41:56.513284 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.513269    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container"
Sep 18 20:41:56.513337 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.513285    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container"
Sep 18 20:41:56.513337 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.513296    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kube-rbac-proxy"
Sep 18 20:41:56.513337 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.513302    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kube-rbac-proxy"
Sep 18 20:41:56.513428 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.513380    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kserve-container"
Sep 18 20:41:56.513428 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.513388    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b117c588-1478-4cf8-8979-c74972d34bca" containerName="kube-rbac-proxy"
Sep 18 20:41:56.516311 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.516296    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.518619 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.518593    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-e7889-predictor-serving-cert\""
Sep 18 20:41:56.518759 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.518625    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-e7889-kube-rbac-proxy-sar-config\""
Sep 18 20:41:56.530271 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.530242    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"]
Sep 18 20:41:56.606308 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.603460    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.606308 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.603556    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.606308 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.603613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkcnr\" (UniqueName: \"kubernetes.io/projected/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-api-access-pkcnr\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.698523 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.698480    2565 generic.go:356] "Generic (PLEG): container finished" podID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerID="4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b" exitCode=2
Sep 18 20:41:56.698724 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.698562    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" event={"ID":"ef5aba97-3d63-4bd9-8090-df52ac3112f9","Type":"ContainerDied","Data":"4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b"}
Sep 18 20:41:56.704881 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.704855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.705016 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.704910    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.705016 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.704951    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkcnr\" (UniqueName: \"kubernetes.io/projected/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-api-access-pkcnr\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.705137 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:41:56.705072    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-e7889-predictor-serving-cert: secret "error-404-isvc-e7889-predictor-serving-cert" not found
Sep 18 20:41:56.705137 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:41:56.705133    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls podName:d909d829-573a-4d5a-861b-9e038c6a9d03 nodeName:}" failed. No retries permitted until 2026-09-18 20:41:57.205113636 +0000 UTC m=+1297.080043694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls") pod "error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" (UID: "d909d829-573a-4d5a-861b-9e038c6a9d03") : secret "error-404-isvc-e7889-predictor-serving-cert" not found
Sep 18 20:41:56.705497 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.705477    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:56.715917 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:56.715895    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkcnr\" (UniqueName: \"kubernetes.io/projected/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-api-access-pkcnr\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:57.209152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:57.209108    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:57.211475 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:57.211453    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls\") pod \"error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") " pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:57.426336 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:57.426304    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:57.698059 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:57.698031    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"]
Sep 18 20:41:57.700534 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:41:57.700506    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd909d829_573a_4d5a_861b_9e038c6a9d03.slice/crio-3d7c317193b3e41e95136383f1af119cbd189c4cff273471dfdcad161ade2163 WatchSource:0}: Error finding container 3d7c317193b3e41e95136383f1af119cbd189c4cff273471dfdcad161ade2163: Status 404 returned error can't find the container with id 3d7c317193b3e41e95136383f1af119cbd189c4cff273471dfdcad161ade2163
Sep 18 20:41:58.710935 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:58.710897    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" event={"ID":"d909d829-573a-4d5a-861b-9e038c6a9d03","Type":"ContainerStarted","Data":"043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956"}
Sep 18 20:41:58.711528 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:58.710956    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" event={"ID":"d909d829-573a-4d5a-861b-9e038c6a9d03","Type":"ContainerStarted","Data":"fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8"}
Sep 18 20:41:58.711528 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:58.710978    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" event={"ID":"d909d829-573a-4d5a-861b-9e038c6a9d03","Type":"ContainerStarted","Data":"3d7c317193b3e41e95136383f1af119cbd189c4cff273471dfdcad161ade2163"}
Sep 18 20:41:58.711528 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:58.711150    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:58.727317 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:58.727271    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podStartSLOduration=2.727256536 podStartE2EDuration="2.727256536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:41:58.726397959 +0000 UTC m=+1298.601328026" watchObservedRunningTime="2026-09-18 20:41:58.727256536 +0000 UTC m=+1298.602186599"
Sep 18 20:41:59.407076 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:59.407032    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.24:8643/healthz\": dial tcp 10.132.0.24:8643: connect: connection refused"
Sep 18 20:41:59.413441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:59.413416    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 18 20:41:59.715575 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:59.715553    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:41:59.716786 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:59.716760    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 18 20:41:59.885409 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:41:59.885386    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:42:00.030625 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.030535    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-rbac-proxy-sar-config\") pod \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") "
Sep 18 20:42:00.030625 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.030585    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6c7r\" (UniqueName: \"kubernetes.io/projected/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-api-access-k6c7r\") pod \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") "
Sep 18 20:42:00.030807 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.030661    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef5aba97-3d63-4bd9-8090-df52ac3112f9-proxy-tls\") pod \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\" (UID: \"ef5aba97-3d63-4bd9-8090-df52ac3112f9\") "
Sep 18 20:42:00.030969 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.030941    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ef5aba97-3d63-4bd9-8090-df52ac3112f9" (UID: "ef5aba97-3d63-4bd9-8090-df52ac3112f9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:42:00.032688 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.032658    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-api-access-k6c7r" (OuterVolumeSpecName: "kube-api-access-k6c7r") pod "ef5aba97-3d63-4bd9-8090-df52ac3112f9" (UID: "ef5aba97-3d63-4bd9-8090-df52ac3112f9"). InnerVolumeSpecName "kube-api-access-k6c7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:42:00.032795 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.032690    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef5aba97-3d63-4bd9-8090-df52ac3112f9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ef5aba97-3d63-4bd9-8090-df52ac3112f9" (UID: "ef5aba97-3d63-4bd9-8090-df52ac3112f9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:00.131289 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.131255    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef5aba97-3d63-4bd9-8090-df52ac3112f9-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:42:00.131289 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.131284    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:42:00.131289 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.131293    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6c7r\" (UniqueName: \"kubernetes.io/projected/ef5aba97-3d63-4bd9-8090-df52ac3112f9-kube-api-access-k6c7r\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:42:00.721084 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.721049    2565 generic.go:356] "Generic (PLEG): container finished" podID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerID="66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6" exitCode=0
Sep 18 20:42:00.721581 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.721101    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"
Sep 18 20:42:00.721581 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.721105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" event={"ID":"ef5aba97-3d63-4bd9-8090-df52ac3112f9","Type":"ContainerDied","Data":"66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6"}
Sep 18 20:42:00.721581 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.721166    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89" event={"ID":"ef5aba97-3d63-4bd9-8090-df52ac3112f9","Type":"ContainerDied","Data":"ab4f9efab663c7c56afd8258b4f08ac5de6a794251878939a09f2440b85bf072"}
Sep 18 20:42:00.721581 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.721193    2565 scope.go:117] "RemoveContainer" containerID="4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b"
Sep 18 20:42:00.721844 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.721820    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 18 20:42:00.729045 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.729008    2565 scope.go:117] "RemoveContainer" containerID="66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6"
Sep 18 20:42:00.736244 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.736222    2565 scope.go:117] "RemoveContainer" containerID="4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b"
Sep 18 20:42:00.736650 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:42:00.736627    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b\": container with ID starting with 4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b not found: ID does not exist" containerID="4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b"
Sep 18 20:42:00.736765 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.736659    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b"} err="failed to get container status \"4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b\": rpc error: code = NotFound desc = could not find container \"4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b\": container with ID starting with 4ed105613c43ad489e977da00300a8eba4784f4bb1beb1a0158c3a5fd077df6b not found: ID does not exist"
Sep 18 20:42:00.736765 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.736679    2565 scope.go:117] "RemoveContainer" containerID="66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6"
Sep 18 20:42:00.737097 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:42:00.737069    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6\": container with ID starting with 66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6 not found: ID does not exist" containerID="66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6"
Sep 18 20:42:00.737178 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.737113    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6"} err="failed to get container status \"66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6\": rpc error: code = NotFound desc = could not find container \"66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6\": container with ID starting with 66fea2769863cbf4f87f93d57963fb0c73a5561faa0ea102aac5288d23b2a2c6 not found: ID does not exist"
Sep 18 20:42:00.738089 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.738063    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"]
Sep 18 20:42:00.741585 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:00.741564    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2bced-predictor-66fb58dcfb-jlz89"]
Sep 18 20:42:02.602949 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:02.602915    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" path="/var/lib/kubelet/pods/ef5aba97-3d63-4bd9-8090-df52ac3112f9/volumes"
Sep 18 20:42:04.592999 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:04.592952    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 18 20:42:05.726160 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:05.726131    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:42:05.726560 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:05.726533    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 18 20:42:14.593157 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:14.593104    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 18 20:42:15.727150 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:15.727110    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 18 20:42:24.594370 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:24.594333    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:42:25.726567 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:25.726528    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 18 20:42:35.726781 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:35.726738    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 18 20:42:45.727370 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:45.727286    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 18 20:42:46.241337 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.241302    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"]
Sep 18 20:42:46.241608 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.241581    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container" containerID="cri-o://6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f" gracePeriod=30
Sep 18 20:42:46.241703 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.241631    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kube-rbac-proxy" containerID="cri-o://8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9" gracePeriod=30
Sep 18 20:42:46.438772 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.438731    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"]
Sep 18 20:42:46.439140 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.439121    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kube-rbac-proxy"
Sep 18 20:42:46.439251 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.439143    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kube-rbac-proxy"
Sep 18 20:42:46.439251 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.439161    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container"
Sep 18 20:42:46.439251 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.439170    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container"
Sep 18 20:42:46.439400 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.439275    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kube-rbac-proxy"
Sep 18 20:42:46.439400 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.439289    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5aba97-3d63-4bd9-8090-df52ac3112f9" containerName="kserve-container"
Sep 18 20:42:46.444780 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.443773    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.446880 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.446855    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4d797-predictor-serving-cert\""
Sep 18 20:42:46.446880 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.446870    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4d797-kube-rbac-proxy-sar-config\""
Sep 18 20:42:46.455602 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.455579    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"]
Sep 18 20:42:46.499477 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.499396    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-proxy-tls\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.499477 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.499436    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8rrf\" (UniqueName: \"kubernetes.io/projected/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-api-access-v8rrf\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.499653 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.499590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.601297 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.601258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.601482 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.601321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-proxy-tls\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.601482 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.601353    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8rrf\" (UniqueName: \"kubernetes.io/projected/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-api-access-v8rrf\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.602081 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.602051    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.604257 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.604235    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-proxy-tls\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.609001 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.608978    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8rrf\" (UniqueName: \"kubernetes.io/projected/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-api-access-v8rrf\") pod \"error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") " pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.755232 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.755132    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:46.917258 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.917221    2565 generic.go:356] "Generic (PLEG): container finished" podID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerID="8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9" exitCode=2
Sep 18 20:42:46.917258 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:46.917237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" event={"ID":"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3","Type":"ContainerDied","Data":"8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9"}
Sep 18 20:42:47.031695 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:47.031667    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"]
Sep 18 20:42:47.033398 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:42:47.033369    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76efc5ca_f01f_414c_8db1_fa51cc51e4ac.slice/crio-0ccb4abf268ff644817ea0139e9858c6a04da2a91fe6e25e0d12039dd4f81291 WatchSource:0}: Error finding container 0ccb4abf268ff644817ea0139e9858c6a04da2a91fe6e25e0d12039dd4f81291: Status 404 returned error can't find the container with id 0ccb4abf268ff644817ea0139e9858c6a04da2a91fe6e25e0d12039dd4f81291
Sep 18 20:42:47.923071 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:47.923041    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" event={"ID":"76efc5ca-f01f-414c-8db1-fa51cc51e4ac","Type":"ContainerStarted","Data":"a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7"}
Sep 18 20:42:47.923491 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:47.923096    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" event={"ID":"76efc5ca-f01f-414c-8db1-fa51cc51e4ac","Type":"ContainerStarted","Data":"c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc"}
Sep 18 20:42:47.923491 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:47.923122    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" event={"ID":"76efc5ca-f01f-414c-8db1-fa51cc51e4ac","Type":"ContainerStarted","Data":"0ccb4abf268ff644817ea0139e9858c6a04da2a91fe6e25e0d12039dd4f81291"}
Sep 18 20:42:47.923491 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:47.923182    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:47.948141 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:47.948090    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podStartSLOduration=1.9480731150000001 podStartE2EDuration="1.948073115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:42:47.94563212 +0000 UTC m=+1347.820562218" watchObservedRunningTime="2026-09-18 20:42:47.948073115 +0000 UTC m=+1347.823003179"
Sep 18 20:42:48.926927 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:48.926896    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:48.928269 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:48.928240    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:42:49.588056 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.588012    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused"
Sep 18 20:42:49.682387 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.682362    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:42:49.730661 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.730632    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js8w6\" (UniqueName: \"kubernetes.io/projected/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-api-access-js8w6\") pod \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") "
Sep 18 20:42:49.730818 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.730670    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-rbac-proxy-sar-config\") pod \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") "
Sep 18 20:42:49.730818 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.730719    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls\") pod \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\" (UID: \"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3\") "
Sep 18 20:42:49.731152 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.731121    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" (UID: "5738ee2e-80d0-48aa-b8fd-11f37f76c1d3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:42:49.732792 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.732774    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" (UID: "5738ee2e-80d0-48aa-b8fd-11f37f76c1d3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:49.732792 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.732786    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-api-access-js8w6" (OuterVolumeSpecName: "kube-api-access-js8w6") pod "5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" (UID: "5738ee2e-80d0-48aa-b8fd-11f37f76c1d3"). InnerVolumeSpecName "kube-api-access-js8w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:42:49.832051 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.831974    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js8w6\" (UniqueName: \"kubernetes.io/projected/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-api-access-js8w6\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:42:49.832051 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.832001    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:42:49.832051 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.832012    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:42:49.931995 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.931954    2565 generic.go:356] "Generic (PLEG): container finished" podID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerID="6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f" exitCode=0
Sep 18 20:42:49.932468 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.932091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" event={"ID":"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3","Type":"ContainerDied","Data":"6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f"}
Sep 18 20:42:49.932468 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.932137    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"
Sep 18 20:42:49.932468 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.932182    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n" event={"ID":"5738ee2e-80d0-48aa-b8fd-11f37f76c1d3","Type":"ContainerDied","Data":"9da3b40fd0015bf29925f277eaa68cde5acbfe5a6e9071f59bd6ff888acef721"}
Sep 18 20:42:49.932468 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.932227    2565 scope.go:117] "RemoveContainer" containerID="8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9"
Sep 18 20:42:49.932845 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.932443    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:42:49.940846 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.940823    2565 scope.go:117] "RemoveContainer" containerID="6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f"
Sep 18 20:42:49.948319 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.948297    2565 scope.go:117] "RemoveContainer" containerID="8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9"
Sep 18 20:42:49.948590 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:42:49.948566    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9\": container with ID starting with 8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9 not found: ID does not exist" containerID="8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9"
Sep 18 20:42:49.948681 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.948597    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9"} err="failed to get container status \"8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9\": rpc error: code = NotFound desc = could not find container \"8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9\": container with ID starting with 8211a783c19e9f5590c7b7cce8fd8bb46535c0cecb53695f743e3b7cb719d1c9 not found: ID does not exist"
Sep 18 20:42:49.948681 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.948614    2565 scope.go:117] "RemoveContainer" containerID="6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f"
Sep 18 20:42:49.948851 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:42:49.948828    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f\": container with ID starting with 6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f not found: ID does not exist" containerID="6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f"
Sep 18 20:42:49.948892 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.948861    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f"} err="failed to get container status \"6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f\": rpc error: code = NotFound desc = could not find container \"6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f\": container with ID starting with 6b8bab72cdb673de86ad6be6a9cee05bbfaaf485f26f606ed7e88b033a78b50f not found: ID does not exist"
Sep 18 20:42:49.953302 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.953280    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"]
Sep 18 20:42:49.957914 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:49.957891    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c092f-predictor-64b7d65596-7k47n"]
Sep 18 20:42:50.600477 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:50.600444    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" path="/var/lib/kubelet/pods/5738ee2e-80d0-48aa-b8fd-11f37f76c1d3/volumes"
Sep 18 20:42:54.937865 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:54.937837    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:42:54.938271 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:54.938231    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:42:55.727377 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:42:55.727346    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:43:04.939264 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:04.939200    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:43:14.938911 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:14.938875    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:43:17.715299 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.715263    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"]
Sep 18 20:43:17.715761 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.715506    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container" containerID="cri-o://fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8" gracePeriod=30
Sep 18 20:43:17.715761 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.715552    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kube-rbac-proxy" containerID="cri-o://043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956" gracePeriod=30
Sep 18 20:43:17.879098 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.879064    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"]
Sep 18 20:43:17.879391 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.879378    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kube-rbac-proxy"
Sep 18 20:43:17.879441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.879392    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kube-rbac-proxy"
Sep 18 20:43:17.879441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.879405    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container"
Sep 18 20:43:17.879441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.879410    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container"
Sep 18 20:43:17.879537 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.879466    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kube-rbac-proxy"
Sep 18 20:43:17.879537 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.879475    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5738ee2e-80d0-48aa-b8fd-11f37f76c1d3" containerName="kserve-container"
Sep 18 20:43:17.884024 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.883994    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:17.886629 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.886603    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8cb8c-predictor-serving-cert\""
Sep 18 20:43:17.886753 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.886633    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8cb8c-kube-rbac-proxy-sar-config\""
Sep 18 20:43:17.894398 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:17.894376    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"]
Sep 18 20:43:18.046663 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.046582    2565 generic.go:356] "Generic (PLEG): container finished" podID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerID="043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956" exitCode=2
Sep 18 20:43:18.046800 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.046656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" event={"ID":"d909d829-573a-4d5a-861b-9e038c6a9d03","Type":"ContainerDied","Data":"043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956"}
Sep 18 20:43:18.054960 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.054937    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-proxy-tls\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.055085 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.054988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.055085 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.055053    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sz2k\" (UniqueName: \"kubernetes.io/projected/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-api-access-5sz2k\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.156360 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.156322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sz2k\" (UniqueName: \"kubernetes.io/projected/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-api-access-5sz2k\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.156533 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.156389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-proxy-tls\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.156533 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.156430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.157074 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.157049    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.158899 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.158869    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-proxy-tls\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.165138 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.165115    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sz2k\" (UniqueName: \"kubernetes.io/projected/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-api-access-5sz2k\") pod \"error-404-isvc-8cb8c-predictor-bdf644456-v5zh5\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") " pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.195147 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.195117    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:18.465221 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:18.465175    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"]
Sep 18 20:43:18.468811 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:43:18.468787    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fac5e6_36ee_43f7_9fa7_936fa18b4f4d.slice/crio-3d5214be8aeff6d12004f2ee7b25d2050b30dd802caed00a45c4f52029fbb628 WatchSource:0}: Error finding container 3d5214be8aeff6d12004f2ee7b25d2050b30dd802caed00a45c4f52029fbb628: Status 404 returned error can't find the container with id 3d5214be8aeff6d12004f2ee7b25d2050b30dd802caed00a45c4f52029fbb628
Sep 18 20:43:19.052628 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:19.052585    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" event={"ID":"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d","Type":"ContainerStarted","Data":"69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe"}
Sep 18 20:43:19.053077 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:19.052653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" event={"ID":"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d","Type":"ContainerStarted","Data":"2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120"}
Sep 18 20:43:19.053077 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:19.052680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" event={"ID":"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d","Type":"ContainerStarted","Data":"3d5214be8aeff6d12004f2ee7b25d2050b30dd802caed00a45c4f52029fbb628"}
Sep 18 20:43:19.053077 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:19.052718    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:19.071059 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:19.071015    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podStartSLOduration=2.071000713 podStartE2EDuration="2.071000713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:43:19.070583647 +0000 UTC m=+1378.945513734" watchObservedRunningTime="2026-09-18 20:43:19.071000713 +0000 UTC m=+1378.945930777"
Sep 18 20:43:20.056537 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:20.056502    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:20.057975 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:20.057946    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:43:20.722715 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:20.722680    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.26:8643/healthz\": dial tcp 10.132.0.26:8643: connect: connection refused"
Sep 18 20:43:21.061417 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.061379    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:43:21.454993 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.454967    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:43:21.585902 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.585811    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls\") pod \"d909d829-573a-4d5a-861b-9e038c6a9d03\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") "
Sep 18 20:43:21.585902 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.585862    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkcnr\" (UniqueName: \"kubernetes.io/projected/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-api-access-pkcnr\") pod \"d909d829-573a-4d5a-861b-9e038c6a9d03\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") "
Sep 18 20:43:21.586123 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.585919    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-rbac-proxy-sar-config\") pod \"d909d829-573a-4d5a-861b-9e038c6a9d03\" (UID: \"d909d829-573a-4d5a-861b-9e038c6a9d03\") "
Sep 18 20:43:21.586363 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.586333    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d909d829-573a-4d5a-861b-9e038c6a9d03" (UID: "d909d829-573a-4d5a-861b-9e038c6a9d03"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:43:21.587951 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.587922    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d909d829-573a-4d5a-861b-9e038c6a9d03" (UID: "d909d829-573a-4d5a-861b-9e038c6a9d03"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:43:21.588088 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.588067    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-api-access-pkcnr" (OuterVolumeSpecName: "kube-api-access-pkcnr") pod "d909d829-573a-4d5a-861b-9e038c6a9d03" (UID: "d909d829-573a-4d5a-861b-9e038c6a9d03"). InnerVolumeSpecName "kube-api-access-pkcnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:43:21.687060 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.687027    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:43:21.687060 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.687053    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d909d829-573a-4d5a-861b-9e038c6a9d03-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:43:21.687060 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:21.687064    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkcnr\" (UniqueName: \"kubernetes.io/projected/d909d829-573a-4d5a-861b-9e038c6a9d03-kube-api-access-pkcnr\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:43:22.066997 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.066960    2565 generic.go:356] "Generic (PLEG): container finished" podID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerID="fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8" exitCode=0
Sep 18 20:43:22.067493 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.067005    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" event={"ID":"d909d829-573a-4d5a-861b-9e038c6a9d03","Type":"ContainerDied","Data":"fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8"}
Sep 18 20:43:22.067493 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.067016    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"
Sep 18 20:43:22.067493 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.067051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6" event={"ID":"d909d829-573a-4d5a-861b-9e038c6a9d03","Type":"ContainerDied","Data":"3d7c317193b3e41e95136383f1af119cbd189c4cff273471dfdcad161ade2163"}
Sep 18 20:43:22.067493 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.067078    2565 scope.go:117] "RemoveContainer" containerID="043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956"
Sep 18 20:43:22.075304 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.075287    2565 scope.go:117] "RemoveContainer" containerID="fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8"
Sep 18 20:43:22.084568 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.083075    2565 scope.go:117] "RemoveContainer" containerID="043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956"
Sep 18 20:43:22.084568 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:43:22.083472    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956\": container with ID starting with 043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956 not found: ID does not exist" containerID="043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956"
Sep 18 20:43:22.084568 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.083502    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956"} err="failed to get container status \"043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956\": rpc error: code = NotFound desc = could not find container \"043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956\": container with ID starting with 043e81a06b324b66072eb08a2c32db5b176eaaed1d37784d9c79cb3c50c58956 not found: ID does not exist"
Sep 18 20:43:22.084568 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.083524    2565 scope.go:117] "RemoveContainer" containerID="fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8"
Sep 18 20:43:22.084568 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:43:22.084499    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8\": container with ID starting with fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8 not found: ID does not exist" containerID="fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8"
Sep 18 20:43:22.084568 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.084537    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8"} err="failed to get container status \"fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8\": rpc error: code = NotFound desc = could not find container \"fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8\": container with ID starting with fa4c7fb5690916231c0966b743a9ad16ac432d6420a6f2419df0756bf3fe5ff8 not found: ID does not exist"
Sep 18 20:43:22.086848 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.086826    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"]
Sep 18 20:43:22.090286 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.090264    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e7889-predictor-6f95cf4c98-8h7w6"]
Sep 18 20:43:22.600172 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:22.600134    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" path="/var/lib/kubelet/pods/d909d829-573a-4d5a-861b-9e038c6a9d03/volumes"
Sep 18 20:43:24.938483 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:24.938448    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:43:26.066381 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:26.066354    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:43:26.066773 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:26.066735    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:43:34.939063 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:34.939017    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:43:36.067064 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:36.067020    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:43:44.940077 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:44.940048    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:43:46.066968 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:46.066933    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:43:56.067490 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:43:56.067450    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:44:06.066925 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:44:06.066885    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:44:16.068420 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:44:16.068345    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:45:22.502496 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:45:22.502466    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:45:22.504563 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:45:22.504540    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:50:22.526408 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:50:22.526376    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:50:22.529687 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:50:22.529661    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:52:11.671237 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.671177    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"]
Sep 18 20:52:11.673774 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.671471    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" containerID="cri-o://c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc" gracePeriod=30
Sep 18 20:52:11.673774 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.671501    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kube-rbac-proxy" containerID="cri-o://a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7" gracePeriod=30
Sep 18 20:52:11.731075 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.731041    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"]
Sep 18 20:52:11.731370 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.731357    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kube-rbac-proxy"
Sep 18 20:52:11.731427 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.731371    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kube-rbac-proxy"
Sep 18 20:52:11.731427 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.731388    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container"
Sep 18 20:52:11.731427 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.731394    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container"
Sep 18 20:52:11.731530 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.731438    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kube-rbac-proxy"
Sep 18 20:52:11.731530 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.731448    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d909d829-573a-4d5a-861b-9e038c6a9d03" containerName="kserve-container"
Sep 18 20:52:11.734367 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.734352    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.736463 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.736440    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-854fa-predictor-serving-cert\""
Sep 18 20:52:11.736542 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.736448    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-854fa-kube-rbac-proxy-sar-config\""
Sep 18 20:52:11.743578 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.743554    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"]
Sep 18 20:52:11.786537 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.786503    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts65x\" (UniqueName: \"kubernetes.io/projected/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-api-access-ts65x\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.786680 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.786557    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.786680 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.786579    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.887413 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.887379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts65x\" (UniqueName: \"kubernetes.io/projected/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-api-access-ts65x\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.887592 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.887437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.887592 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.887471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.887592 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:52:11.887533    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-854fa-predictor-serving-cert: secret "error-404-isvc-854fa-predictor-serving-cert" not found
Sep 18 20:52:11.887767 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:52:11.887613    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls podName:5e6eb230-f0b0-4942-9b74-e7f13d116ebf nodeName:}" failed. No retries permitted until 2026-09-18 20:52:12.387594101 +0000 UTC m=+1912.262524148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls") pod "error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" (UID: "5e6eb230-f0b0-4942-9b74-e7f13d116ebf") : secret "error-404-isvc-854fa-predictor-serving-cert" not found
Sep 18 20:52:11.888113 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.888090    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:11.895688 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:11.895663    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts65x\" (UniqueName: \"kubernetes.io/projected/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-api-access-ts65x\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:12.228481 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:12.228448    2565 generic.go:356] "Generic (PLEG): container finished" podID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerID="a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7" exitCode=2
Sep 18 20:52:12.228652 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:12.228505    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" event={"ID":"76efc5ca-f01f-414c-8db1-fa51cc51e4ac","Type":"ContainerDied","Data":"a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7"}
Sep 18 20:52:12.390741 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:12.390702    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:12.393125 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:12.393098    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls\") pod \"error-404-isvc-854fa-predictor-5f46c54d88-kcp7l\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") " pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:12.645168 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:12.645088    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:12.916140 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:12.916109    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"]
Sep 18 20:52:12.918640 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:52:12.918617    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6eb230_f0b0_4942_9b74_e7f13d116ebf.slice/crio-2a544ba90bfca379fe3addd2364e8e8d6f6537e225affe0fb460e14ff98cd8f3 WatchSource:0}: Error finding container 2a544ba90bfca379fe3addd2364e8e8d6f6537e225affe0fb460e14ff98cd8f3: Status 404 returned error can't find the container with id 2a544ba90bfca379fe3addd2364e8e8d6f6537e225affe0fb460e14ff98cd8f3
Sep 18 20:52:13.234392 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:13.234351    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" event={"ID":"5e6eb230-f0b0-4942-9b74-e7f13d116ebf","Type":"ContainerStarted","Data":"983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6"}
Sep 18 20:52:13.234588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:13.234415    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" event={"ID":"5e6eb230-f0b0-4942-9b74-e7f13d116ebf","Type":"ContainerStarted","Data":"04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182"}
Sep 18 20:52:13.234588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:13.234443    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" event={"ID":"5e6eb230-f0b0-4942-9b74-e7f13d116ebf","Type":"ContainerStarted","Data":"2a544ba90bfca379fe3addd2364e8e8d6f6537e225affe0fb460e14ff98cd8f3"}
Sep 18 20:52:13.234588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:13.234468    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:13.234588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:13.234493    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:13.235904 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:13.235881    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 18 20:52:14.238328 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:14.238283    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 18 20:52:14.932945 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:14.932901    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused"
Sep 18 20:52:14.939309 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:14.939286    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 18 20:52:15.116231 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.115968    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:52:15.135171 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.135122    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podStartSLOduration=4.13510702 podStartE2EDuration="4.13510702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:52:13.249759843 +0000 UTC m=+1913.124689928" watchObservedRunningTime="2026-09-18 20:52:15.13510702 +0000 UTC m=+1915.010037083"
Sep 18 20:52:15.209667 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.209637    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8rrf\" (UniqueName: \"kubernetes.io/projected/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-api-access-v8rrf\") pod \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") "
Sep 18 20:52:15.209819 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.209716    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-proxy-tls\") pod \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") "
Sep 18 20:52:15.209819 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.209749    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-rbac-proxy-sar-config\") pod \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\" (UID: \"76efc5ca-f01f-414c-8db1-fa51cc51e4ac\") "
Sep 18 20:52:15.210153 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.210124    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "76efc5ca-f01f-414c-8db1-fa51cc51e4ac" (UID: "76efc5ca-f01f-414c-8db1-fa51cc51e4ac"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:52:15.211730 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.211700    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-api-access-v8rrf" (OuterVolumeSpecName: "kube-api-access-v8rrf") pod "76efc5ca-f01f-414c-8db1-fa51cc51e4ac" (UID: "76efc5ca-f01f-414c-8db1-fa51cc51e4ac"). InnerVolumeSpecName "kube-api-access-v8rrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:52:15.211807 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.211736    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "76efc5ca-f01f-414c-8db1-fa51cc51e4ac" (UID: "76efc5ca-f01f-414c-8db1-fa51cc51e4ac"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:52:15.245561 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.245533    2565 generic.go:356] "Generic (PLEG): container finished" podID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerID="c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc" exitCode=0
Sep 18 20:52:15.245867 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.245592    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"
Sep 18 20:52:15.245867 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.245597    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" event={"ID":"76efc5ca-f01f-414c-8db1-fa51cc51e4ac","Type":"ContainerDied","Data":"c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc"}
Sep 18 20:52:15.245867 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.245680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf" event={"ID":"76efc5ca-f01f-414c-8db1-fa51cc51e4ac","Type":"ContainerDied","Data":"0ccb4abf268ff644817ea0139e9858c6a04da2a91fe6e25e0d12039dd4f81291"}
Sep 18 20:52:15.245867 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.245706    2565 scope.go:117] "RemoveContainer" containerID="a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7"
Sep 18 20:52:15.253857 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.253838    2565 scope.go:117] "RemoveContainer" containerID="c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc"
Sep 18 20:52:15.261036 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.261021    2565 scope.go:117] "RemoveContainer" containerID="a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7"
Sep 18 20:52:15.261299 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:52:15.261272    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7\": container with ID starting with a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7 not found: ID does not exist" containerID="a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7"
Sep 18 20:52:15.261345 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.261307    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7"} err="failed to get container status \"a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7\": rpc error: code = NotFound desc = could not find container \"a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7\": container with ID starting with a321085602010fcb5525b150940baba49adcdf7b58d6cab657e28ecf4c1796d7 not found: ID does not exist"
Sep 18 20:52:15.261345 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.261322    2565 scope.go:117] "RemoveContainer" containerID="c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc"
Sep 18 20:52:15.261548 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:52:15.261533    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc\": container with ID starting with c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc not found: ID does not exist" containerID="c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc"
Sep 18 20:52:15.261587 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.261554    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc"} err="failed to get container status \"c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc\": rpc error: code = NotFound desc = could not find container \"c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc\": container with ID starting with c874d90cb006003ebba84989ceb5fb28a48d156199248aff0bfa881b645e64bc not found: ID does not exist"
Sep 18 20:52:15.265040 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.265019    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"]
Sep 18 20:52:15.269142 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.269123    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4d797-predictor-5b98b8c79d-hfnzf"]
Sep 18 20:52:15.311157 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.311131    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8rrf\" (UniqueName: \"kubernetes.io/projected/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-api-access-v8rrf\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:52:15.311157 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.311155    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:52:15.311295 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:15.311168    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76efc5ca-f01f-414c-8db1-fa51cc51e4ac-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:52:16.600273 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:16.600235    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" path="/var/lib/kubelet/pods/76efc5ca-f01f-414c-8db1-fa51cc51e4ac/volumes"
Sep 18 20:52:19.247382 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:19.247352    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:52:19.247810 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:19.247783    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 18 20:52:29.248352 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:29.248307    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 18 20:52:39.248431 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:39.248389    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 18 20:52:43.095375 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.095336    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"]
Sep 18 20:52:43.095834 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.095675    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" containerID="cri-o://2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120" gracePeriod=30
Sep 18 20:52:43.095834 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.095705    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kube-rbac-proxy" containerID="cri-o://69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe" gracePeriod=30
Sep 18 20:52:43.309003 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.308969    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"]
Sep 18 20:52:43.309294 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.309274    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container"
Sep 18 20:52:43.309346 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.309297    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container"
Sep 18 20:52:43.309346 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.309315    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kube-rbac-proxy"
Sep 18 20:52:43.309346 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.309321    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kube-rbac-proxy"
Sep 18 20:52:43.309445 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.309380    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kube-rbac-proxy"
Sep 18 20:52:43.309445 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.309389    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="76efc5ca-f01f-414c-8db1-fa51cc51e4ac" containerName="kserve-container"
Sep 18 20:52:43.312318 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.312302    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.314408 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.314382    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c7ffd-predictor-serving-cert\""
Sep 18 20:52:43.315465 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.315443    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c7ffd-kube-rbac-proxy-sar-config\""
Sep 18 20:52:43.323928 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.323904    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"]
Sep 18 20:52:43.361085 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.361021    2565 generic.go:356] "Generic (PLEG): container finished" podID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerID="69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe" exitCode=2
Sep 18 20:52:43.361196 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.361089    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" event={"ID":"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d","Type":"ContainerDied","Data":"69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe"}
Sep 18 20:52:43.429577 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.429543    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz986\" (UniqueName: \"kubernetes.io/projected/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-api-access-pz986\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.429729 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.429584    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f410f50a-2558-4e28-9da8-1285e20b4c9f-proxy-tls\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.429729 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.429657    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.530753 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.530720    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz986\" (UniqueName: \"kubernetes.io/projected/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-api-access-pz986\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.530934 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.530759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f410f50a-2558-4e28-9da8-1285e20b4c9f-proxy-tls\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.530934 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.530794    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.531425 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.531403    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.533135 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.533113    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f410f50a-2558-4e28-9da8-1285e20b4c9f-proxy-tls\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.538683 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.538666    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz986\" (UniqueName: \"kubernetes.io/projected/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-api-access-pz986\") pod \"error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") " pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.621418 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.621323    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:43.895075 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:43.895047    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"]
Sep 18 20:52:43.898323 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:52:43.898292    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf410f50a_2558_4e28_9da8_1285e20b4c9f.slice/crio-8a4e5567b3274cbc322824114571d3c0273d512f5f2d43a556337941968cd3fc WatchSource:0}: Error finding container 8a4e5567b3274cbc322824114571d3c0273d512f5f2d43a556337941968cd3fc: Status 404 returned error can't find the container with id 8a4e5567b3274cbc322824114571d3c0273d512f5f2d43a556337941968cd3fc
Sep 18 20:52:44.368433 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:44.368397    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" event={"ID":"f410f50a-2558-4e28-9da8-1285e20b4c9f","Type":"ContainerStarted","Data":"8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91"}
Sep 18 20:52:44.368831 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:44.368461    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" event={"ID":"f410f50a-2558-4e28-9da8-1285e20b4c9f","Type":"ContainerStarted","Data":"78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219"}
Sep 18 20:52:44.368831 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:44.368487    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:44.368831 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:44.368517    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:44.368831 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:44.368541    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" event={"ID":"f410f50a-2558-4e28-9da8-1285e20b4c9f","Type":"ContainerStarted","Data":"8a4e5567b3274cbc322824114571d3c0273d512f5f2d43a556337941968cd3fc"}
Sep 18 20:52:44.370063 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:44.370034    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Sep 18 20:52:44.387686 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:44.387642    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podStartSLOduration=1.387628382 podStartE2EDuration="1.387628382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:52:44.386419751 +0000 UTC m=+1944.261349833" watchObservedRunningTime="2026-09-18 20:52:44.387628382 +0000 UTC m=+1944.262558446"
Sep 18 20:52:45.373017 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:45.372981    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Sep 18 20:52:46.062234 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.062166    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused"
Sep 18 20:52:46.067443 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.067415    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 18 20:52:46.548939 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.548911    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:52:46.654717 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.654631    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-proxy-tls\") pod \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") "
Sep 18 20:52:46.654717 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.654683    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-rbac-proxy-sar-config\") pod \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") "
Sep 18 20:52:46.654717 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.654707    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sz2k\" (UniqueName: \"kubernetes.io/projected/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-api-access-5sz2k\") pod \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\" (UID: \"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d\") "
Sep 18 20:52:46.655097 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.655066    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" (UID: "03fac5e6-36ee-43f7-9fa7-936fa18b4f4d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:52:46.656845 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.656817    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" (UID: "03fac5e6-36ee-43f7-9fa7-936fa18b4f4d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:52:46.656845 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.656815    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-api-access-5sz2k" (OuterVolumeSpecName: "kube-api-access-5sz2k") pod "03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" (UID: "03fac5e6-36ee-43f7-9fa7-936fa18b4f4d"). InnerVolumeSpecName "kube-api-access-5sz2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:52:46.755588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.755560    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:52:46.755588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.755586    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:52:46.755588 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:46.755595    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sz2k\" (UniqueName: \"kubernetes.io/projected/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d-kube-api-access-5sz2k\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:52:47.383111 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.383073    2565 generic.go:356] "Generic (PLEG): container finished" podID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerID="2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120" exitCode=0
Sep 18 20:52:47.383293 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.383129    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"
Sep 18 20:52:47.383293 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.383153    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" event={"ID":"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d","Type":"ContainerDied","Data":"2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120"}
Sep 18 20:52:47.383293 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.383237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5" event={"ID":"03fac5e6-36ee-43f7-9fa7-936fa18b4f4d","Type":"ContainerDied","Data":"3d5214be8aeff6d12004f2ee7b25d2050b30dd802caed00a45c4f52029fbb628"}
Sep 18 20:52:47.383293 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.383268    2565 scope.go:117] "RemoveContainer" containerID="69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe"
Sep 18 20:52:47.391598 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.391580    2565 scope.go:117] "RemoveContainer" containerID="2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120"
Sep 18 20:52:47.400311 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.400291    2565 scope.go:117] "RemoveContainer" containerID="69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe"
Sep 18 20:52:47.400620 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:52:47.400604    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe\": container with ID starting with 69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe not found: ID does not exist" containerID="69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe"
Sep 18 20:52:47.400672 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.400628    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe"} err="failed to get container status \"69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe\": rpc error: code = NotFound desc = could not find container \"69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe\": container with ID starting with 69c660a6020867c6c39993639163ba88949643203cf13770140b7ece7f00c5fe not found: ID does not exist"
Sep 18 20:52:47.400672 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.400643    2565 scope.go:117] "RemoveContainer" containerID="2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120"
Sep 18 20:52:47.400889 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:52:47.400874    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120\": container with ID starting with 2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120 not found: ID does not exist" containerID="2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120"
Sep 18 20:52:47.400942 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.400891    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120"} err="failed to get container status \"2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120\": rpc error: code = NotFound desc = could not find container \"2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120\": container with ID starting with 2d5b7c2c12d8e2bb2982d6a5bdb233f230c9db1c9819d0fbc32d9eb8340f8120 not found: ID does not exist"
Sep 18 20:52:47.404202 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.404182    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"]
Sep 18 20:52:47.408500 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:47.408483    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8cb8c-predictor-bdf644456-v5zh5"]
Sep 18 20:52:48.601740 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:48.601697    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" path="/var/lib/kubelet/pods/03fac5e6-36ee-43f7-9fa7-936fa18b4f4d/volumes"
Sep 18 20:52:49.248516 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:49.248478    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 18 20:52:50.377670 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:50.377639    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:52:50.378337 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:50.378304    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Sep 18 20:52:59.247980 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:52:59.247943    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 18 20:53:00.379093 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:00.379051    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Sep 18 20:53:09.249349 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:09.249313    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:53:10.378319 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:10.378281    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Sep 18 20:53:20.379074 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:20.378993    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Sep 18 20:53:30.378809 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:30.378771    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.30:8080: connect: connection refused"
Sep 18 20:53:32.742734 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.742693    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"]
Sep 18 20:53:32.743289 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.742945    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container" containerID="cri-o://04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182" gracePeriod=30
Sep 18 20:53:32.743289 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.742987    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kube-rbac-proxy" containerID="cri-o://983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6" gracePeriod=30
Sep 18 20:53:32.952134 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.952098    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"]
Sep 18 20:53:32.952480 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.952465    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kube-rbac-proxy"
Sep 18 20:53:32.952534 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.952482    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kube-rbac-proxy"
Sep 18 20:53:32.952534 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.952494    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container"
Sep 18 20:53:32.952534 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.952500    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container"
Sep 18 20:53:32.952635 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.952548    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kserve-container"
Sep 18 20:53:32.952635 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.952556    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="03fac5e6-36ee-43f7-9fa7-936fa18b4f4d" containerName="kube-rbac-proxy"
Sep 18 20:53:32.955475 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.955459    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:32.957770 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.957744    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8bda6-predictor-serving-cert\""
Sep 18 20:53:32.957770 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.957752    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8bda6-kube-rbac-proxy-sar-config\""
Sep 18 20:53:32.971441 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:32.971413    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"]
Sep 18 20:53:33.013895 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.013806    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.013895 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.013867    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfpfj\" (UniqueName: \"kubernetes.io/projected/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-api-access-pfpfj\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.013895 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.013898    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.114485 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.114446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfpfj\" (UniqueName: \"kubernetes.io/projected/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-api-access-pfpfj\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.114671 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.114493    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.114671 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.114528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.114671 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:53:33.114629    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-serving-cert: secret "error-404-isvc-8bda6-predictor-serving-cert" not found
Sep 18 20:53:33.114818 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:53:33.114677    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls podName:52585ff4-b97a-40ac-a56c-fc2ff98ce90f nodeName:}" failed. No retries permitted until 2026-09-18 20:53:33.614662272 +0000 UTC m=+1993.489592313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls") pod "error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" (UID: "52585ff4-b97a-40ac-a56c-fc2ff98ce90f") : secret "error-404-isvc-8bda6-predictor-serving-cert" not found
Sep 18 20:53:33.115178 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.115156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.124722 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.124691    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfpfj\" (UniqueName: \"kubernetes.io/projected/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-api-access-pfpfj\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.579250 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.579192    2565 generic.go:356] "Generic (PLEG): container finished" podID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerID="983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6" exitCode=2
Sep 18 20:53:33.579417 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.579253    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" event={"ID":"5e6eb230-f0b0-4942-9b74-e7f13d116ebf","Type":"ContainerDied","Data":"983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6"}
Sep 18 20:53:33.619256 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.619224    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.621711 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.621678    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls\") pod \"error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") " pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:33.865943 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:33.865857    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:34.134009 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:34.133875    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"]
Sep 18 20:53:34.136638 ip-10-0-137-48 kubenswrapper[2565]: W0918 20:53:34.136612    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52585ff4_b97a_40ac_a56c_fc2ff98ce90f.slice/crio-9dd16675a8d7e4de7f74ee207ee7e876d5401a24c60ba6f1b9ee65420f32593c WatchSource:0}: Error finding container 9dd16675a8d7e4de7f74ee207ee7e876d5401a24c60ba6f1b9ee65420f32593c: Status 404 returned error can't find the container with id 9dd16675a8d7e4de7f74ee207ee7e876d5401a24c60ba6f1b9ee65420f32593c
Sep 18 20:53:34.239243 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:34.239187    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.29:8643/healthz\": dial tcp 10.132.0.29:8643: connect: connection refused"
Sep 18 20:53:34.584911 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:34.584871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" event={"ID":"52585ff4-b97a-40ac-a56c-fc2ff98ce90f","Type":"ContainerStarted","Data":"c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa"}
Sep 18 20:53:34.585110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:34.584931    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" event={"ID":"52585ff4-b97a-40ac-a56c-fc2ff98ce90f","Type":"ContainerStarted","Data":"cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c"}
Sep 18 20:53:34.585110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:34.584955    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:34.585110 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:34.584973    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" event={"ID":"52585ff4-b97a-40ac-a56c-fc2ff98ce90f","Type":"ContainerStarted","Data":"9dd16675a8d7e4de7f74ee207ee7e876d5401a24c60ba6f1b9ee65420f32593c"}
Sep 18 20:53:34.602848 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:34.602795    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podStartSLOduration=2.6027794010000003 podStartE2EDuration="2.602779401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:53:34.601624617 +0000 UTC m=+1994.476554684" watchObservedRunningTime="2026-09-18 20:53:34.602779401 +0000 UTC m=+1994.477709465"
Sep 18 20:53:35.589660 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:35.589626    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:35.591119 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:35.591089    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 18 20:53:36.284178 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.284154    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:53:36.339178 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.339145    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-rbac-proxy-sar-config\") pod \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") "
Sep 18 20:53:36.339352 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.339233    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts65x\" (UniqueName: \"kubernetes.io/projected/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-api-access-ts65x\") pod \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") "
Sep 18 20:53:36.339352 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.339258    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls\") pod \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\" (UID: \"5e6eb230-f0b0-4942-9b74-e7f13d116ebf\") "
Sep 18 20:53:36.339563 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.339543    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5e6eb230-f0b0-4942-9b74-e7f13d116ebf" (UID: "5e6eb230-f0b0-4942-9b74-e7f13d116ebf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:53:36.341349 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.341322    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5e6eb230-f0b0-4942-9b74-e7f13d116ebf" (UID: "5e6eb230-f0b0-4942-9b74-e7f13d116ebf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:53:36.341429 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.341349    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-api-access-ts65x" (OuterVolumeSpecName: "kube-api-access-ts65x") pod "5e6eb230-f0b0-4942-9b74-e7f13d116ebf" (UID: "5e6eb230-f0b0-4942-9b74-e7f13d116ebf"). InnerVolumeSpecName "kube-api-access-ts65x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:53:36.439893 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.439857    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts65x\" (UniqueName: \"kubernetes.io/projected/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-api-access-ts65x\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:53:36.439893 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.439890    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:53:36.440077 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.439902    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6eb230-f0b0-4942-9b74-e7f13d116ebf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 20:53:36.595964 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.595930    2565 generic.go:356] "Generic (PLEG): container finished" podID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerID="04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182" exitCode=0
Sep 18 20:53:36.596469 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.596049    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"
Sep 18 20:53:36.596713 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.596684    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 18 20:53:36.600440 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.600417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" event={"ID":"5e6eb230-f0b0-4942-9b74-e7f13d116ebf","Type":"ContainerDied","Data":"04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182"}
Sep 18 20:53:36.600576 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.600461    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l" event={"ID":"5e6eb230-f0b0-4942-9b74-e7f13d116ebf","Type":"ContainerDied","Data":"2a544ba90bfca379fe3addd2364e8e8d6f6537e225affe0fb460e14ff98cd8f3"}
Sep 18 20:53:36.600576 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.600496    2565 scope.go:117] "RemoveContainer" containerID="983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6"
Sep 18 20:53:36.610276 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.610261    2565 scope.go:117] "RemoveContainer" containerID="04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182"
Sep 18 20:53:36.618246 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.618229    2565 scope.go:117] "RemoveContainer" containerID="983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6"
Sep 18 20:53:36.618512 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:53:36.618495    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6\": container with ID starting with 983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6 not found: ID does not exist" containerID="983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6"
Sep 18 20:53:36.618560 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.618520    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6"} err="failed to get container status \"983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6\": rpc error: code = NotFound desc = could not find container \"983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6\": container with ID starting with 983145ba4a259e8ee00337f22f09d0db35e180463cf717fcf7a5912caa362ff6 not found: ID does not exist"
Sep 18 20:53:36.618560 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.618537    2565 scope.go:117] "RemoveContainer" containerID="04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182"
Sep 18 20:53:36.618655 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.618633    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"]
Sep 18 20:53:36.618788 ip-10-0-137-48 kubenswrapper[2565]: E0918 20:53:36.618773    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182\": container with ID starting with 04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182 not found: ID does not exist" containerID="04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182"
Sep 18 20:53:36.618830 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.618791    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182"} err="failed to get container status \"04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182\": rpc error: code = NotFound desc = could not find container \"04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182\": container with ID starting with 04227c759b1b48e463c812699fdc676ec3f6730712651ae6ae1e56781e67e182 not found: ID does not exist"
Sep 18 20:53:36.622834 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:36.622813    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-854fa-predictor-5f46c54d88-kcp7l"]
Sep 18 20:53:38.601806 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:38.601775    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" path="/var/lib/kubelet/pods/5e6eb230-f0b0-4942-9b74-e7f13d116ebf/volumes"
Sep 18 20:53:40.379327 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:40.379298    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 20:53:41.601225 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:41.601178    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:53:41.601778 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:41.601751    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 18 20:53:51.602013 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:53:51.601977    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 18 20:54:01.602116 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:54:01.602065    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 18 20:54:11.602004 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:54:11.601955    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 18 20:54:21.602545 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:54:21.602504    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 18 20:54:31.603372 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:54:31.603345    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 20:55:22.550696 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:55:22.550663    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 20:55:22.555252 ip-10-0-137-48 kubenswrapper[2565]: I0918 20:55:22.555228    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:00:22.575623 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:00:22.575594    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:00:22.579782 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:00:22.579760    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:02:58.078549 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:02:58.078506    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"]
Sep 18 21:02:58.079163 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:02:58.078771    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" containerID="cri-o://cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c" gracePeriod=30
Sep 18 21:02:58.079163 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:02:58.078778    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kube-rbac-proxy" containerID="cri-o://c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa" gracePeriod=30
Sep 18 21:02:58.878722 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:02:58.878690    2565 generic.go:356] "Generic (PLEG): container finished" podID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerID="c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa" exitCode=2
Sep 18 21:02:58.878888 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:02:58.878764    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" event={"ID":"52585ff4-b97a-40ac-a56c-fc2ff98ce90f","Type":"ContainerDied","Data":"c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa"}
Sep 18 21:03:01.618544 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.618518    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 21:03:01.685402 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.685376    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls\") pod \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") "
Sep 18 21:03:01.685586 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.685442    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfpfj\" (UniqueName: \"kubernetes.io/projected/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-api-access-pfpfj\") pod \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") "
Sep 18 21:03:01.685586 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.685461    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-rbac-proxy-sar-config\") pod \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\" (UID: \"52585ff4-b97a-40ac-a56c-fc2ff98ce90f\") "
Sep 18 21:03:01.685823 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.685799    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "52585ff4-b97a-40ac-a56c-fc2ff98ce90f" (UID: "52585ff4-b97a-40ac-a56c-fc2ff98ce90f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 21:03:01.687480 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.687459    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "52585ff4-b97a-40ac-a56c-fc2ff98ce90f" (UID: "52585ff4-b97a-40ac-a56c-fc2ff98ce90f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 21:03:01.687555 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.687482    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-api-access-pfpfj" (OuterVolumeSpecName: "kube-api-access-pfpfj") pod "52585ff4-b97a-40ac-a56c-fc2ff98ce90f" (UID: "52585ff4-b97a-40ac-a56c-fc2ff98ce90f"). InnerVolumeSpecName "kube-api-access-pfpfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 21:03:01.785944 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.785907    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfpfj\" (UniqueName: \"kubernetes.io/projected/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-api-access-pfpfj\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 21:03:01.785944 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.785939    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 21:03:01.785944 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.785949    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/52585ff4-b97a-40ac-a56c-fc2ff98ce90f-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 21:03:01.892482 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.892447    2565 generic.go:356] "Generic (PLEG): container finished" podID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerID="cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c" exitCode=0
Sep 18 21:03:01.892659 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.892494    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"
Sep 18 21:03:01.892659 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.892508    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" event={"ID":"52585ff4-b97a-40ac-a56c-fc2ff98ce90f","Type":"ContainerDied","Data":"cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c"}
Sep 18 21:03:01.892659 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.892557    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" event={"ID":"52585ff4-b97a-40ac-a56c-fc2ff98ce90f","Type":"ContainerDied","Data":"9dd16675a8d7e4de7f74ee207ee7e876d5401a24c60ba6f1b9ee65420f32593c"}
Sep 18 21:03:01.892659 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.892586    2565 scope.go:117] "RemoveContainer" containerID="c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa"
Sep 18 21:03:01.901075 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.901058    2565 scope.go:117] "RemoveContainer" containerID="cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c"
Sep 18 21:03:01.907870 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.907853    2565 scope.go:117] "RemoveContainer" containerID="c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa"
Sep 18 21:03:01.908127 ip-10-0-137-48 kubenswrapper[2565]: E0918 21:03:01.908110    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa\": container with ID starting with c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa not found: ID does not exist" containerID="c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa"
Sep 18 21:03:01.908168 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.908136    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa"} err="failed to get container status \"c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa\": rpc error: code = NotFound desc = could not find container \"c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa\": container with ID starting with c0832e2b1312b0091c61693ab3d3516595eca5f1361f94328273360c3cd996aa not found: ID does not exist"
Sep 18 21:03:01.908168 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.908151    2565 scope.go:117] "RemoveContainer" containerID="cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c"
Sep 18 21:03:01.908405 ip-10-0-137-48 kubenswrapper[2565]: E0918 21:03:01.908387    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c\": container with ID starting with cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c not found: ID does not exist" containerID="cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c"
Sep 18 21:03:01.908491 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.908413    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c"} err="failed to get container status \"cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c\": rpc error: code = NotFound desc = could not find container \"cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c\": container with ID starting with cef1e9dddfec08e0daa8b1e1dee81a672a6c00582ab33fa2bb2c25b5852cdd4c not found: ID does not exist"
Sep 18 21:03:01.911692 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.911668    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"]
Sep 18 21:03:01.915866 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:01.915844    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt"]
Sep 18 21:03:02.596861 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:02.596812    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": context deadline exceeded"
Sep 18 21:03:02.599776 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:02.599750    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" path="/var/lib/kubelet/pods/52585ff4-b97a-40ac-a56c-fc2ff98ce90f/volumes"
Sep 18 21:03:02.601986 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:03:02.601961    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8bda6-predictor-69655b5cb6-9wgtt" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: i/o timeout"
Sep 18 21:05:22.598498 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:05:22.598424    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:05:22.604482 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:05:22.604457    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:10:13.714300 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:13.714199    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"]
Sep 18 21:10:13.714832 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:13.714450    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container" containerID="cri-o://78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219" gracePeriod=30
Sep 18 21:10:13.714832 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:13.714526    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kube-rbac-proxy" containerID="cri-o://8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91" gracePeriod=30
Sep 18 21:10:14.617579 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:14.617546    2565 generic.go:356] "Generic (PLEG): container finished" podID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerID="8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91" exitCode=2
Sep 18 21:10:14.617757 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:14.617622    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" event={"ID":"f410f50a-2558-4e28-9da8-1285e20b4c9f","Type":"ContainerDied","Data":"8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91"}
Sep 18 21:10:15.374262 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:15.374194    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.30:8643/healthz\": dial tcp 10.132.0.30:8643: connect: connection refused"
Sep 18 21:10:16.849508 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:16.849483    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 21:10:16.932193 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:16.932159    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f410f50a-2558-4e28-9da8-1285e20b4c9f-proxy-tls\") pod \"f410f50a-2558-4e28-9da8-1285e20b4c9f\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") "
Sep 18 21:10:16.932398 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:16.932226    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz986\" (UniqueName: \"kubernetes.io/projected/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-api-access-pz986\") pod \"f410f50a-2558-4e28-9da8-1285e20b4c9f\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") "
Sep 18 21:10:16.932398 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:16.932267    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-rbac-proxy-sar-config\") pod \"f410f50a-2558-4e28-9da8-1285e20b4c9f\" (UID: \"f410f50a-2558-4e28-9da8-1285e20b4c9f\") "
Sep 18 21:10:16.932702 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:16.932676    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f410f50a-2558-4e28-9da8-1285e20b4c9f" (UID: "f410f50a-2558-4e28-9da8-1285e20b4c9f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 21:10:16.934245 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:16.934225    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f410f50a-2558-4e28-9da8-1285e20b4c9f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f410f50a-2558-4e28-9da8-1285e20b4c9f" (UID: "f410f50a-2558-4e28-9da8-1285e20b4c9f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 21:10:16.934454 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:16.934432    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-api-access-pz986" (OuterVolumeSpecName: "kube-api-access-pz986") pod "f410f50a-2558-4e28-9da8-1285e20b4c9f" (UID: "f410f50a-2558-4e28-9da8-1285e20b4c9f"). InnerVolumeSpecName "kube-api-access-pz986". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 21:10:17.033277 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.033181    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f410f50a-2558-4e28-9da8-1285e20b4c9f-proxy-tls\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 21:10:17.033277 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.033228    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz986\" (UniqueName: \"kubernetes.io/projected/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-api-access-pz986\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 21:10:17.033277 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.033237    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f410f50a-2558-4e28-9da8-1285e20b4c9f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-48.ec2.internal\" DevicePath \"\""
Sep 18 21:10:17.630071 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.630037    2565 generic.go:356] "Generic (PLEG): container finished" podID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerID="78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219" exitCode=0
Sep 18 21:10:17.630275 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.630081    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"
Sep 18 21:10:17.630275 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.630080    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" event={"ID":"f410f50a-2558-4e28-9da8-1285e20b4c9f","Type":"ContainerDied","Data":"78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219"}
Sep 18 21:10:17.630275 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.630236    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br" event={"ID":"f410f50a-2558-4e28-9da8-1285e20b4c9f","Type":"ContainerDied","Data":"8a4e5567b3274cbc322824114571d3c0273d512f5f2d43a556337941968cd3fc"}
Sep 18 21:10:17.630440 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.630277    2565 scope.go:117] "RemoveContainer" containerID="8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91"
Sep 18 21:10:17.638705 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.638691    2565 scope.go:117] "RemoveContainer" containerID="78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219"
Sep 18 21:10:17.645655 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.645636    2565 scope.go:117] "RemoveContainer" containerID="8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91"
Sep 18 21:10:17.645916 ip-10-0-137-48 kubenswrapper[2565]: E0918 21:10:17.645896    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91\": container with ID starting with 8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91 not found: ID does not exist" containerID="8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91"
Sep 18 21:10:17.645977 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.645923    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91"} err="failed to get container status \"8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91\": rpc error: code = NotFound desc = could not find container \"8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91\": container with ID starting with 8a2bce1f1d023ac71a7db7dfa5556c4e0fa1e7f8b55db72d46f1624f82b2bd91 not found: ID does not exist"
Sep 18 21:10:17.645977 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.645941    2565 scope.go:117] "RemoveContainer" containerID="78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219"
Sep 18 21:10:17.646196 ip-10-0-137-48 kubenswrapper[2565]: E0918 21:10:17.646177    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219\": container with ID starting with 78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219 not found: ID does not exist" containerID="78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219"
Sep 18 21:10:17.646270 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.646202    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219"} err="failed to get container status \"78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219\": rpc error: code = NotFound desc = could not find container \"78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219\": container with ID starting with 78d299174bae68f9e3d073dc0cb23e0adbb486846ff947bcdc342a956b0c3219 not found: ID does not exist"
Sep 18 21:10:17.672892 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.672866    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"]
Sep 18 21:10:17.684266 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:17.684238    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c7ffd-predictor-6597cccd7b-mn6br"]
Sep 18 21:10:18.599610 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:18.599575    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" path="/var/lib/kubelet/pods/f410f50a-2558-4e28-9da8-1285e20b4c9f/volumes"
Sep 18 21:10:22.623337 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:22.623190    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:10:22.628933 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:22.628915    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:10:49.205425 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:49.205395    2565 ???:1] "http: TLS handshake error from 10.0.136.235:36464: EOF"
Sep 18 21:10:49.209773 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:49.209748    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qp8n2_0d1d26c8-baa0-4bc3-97bf-ed62049a59f5/global-pull-secret-syncer/0.log"
Sep 18 21:10:49.356487 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:49.356460    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fgtsn_a723e0f7-c700-46c0-ae2a-73ce8f59dff9/konnectivity-agent/0.log"
Sep 18 21:10:49.464957 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:49.464882    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-48.ec2.internal_2855f1593c0393be173039d69f8435c0/haproxy/0.log"
Sep 18 21:10:53.845846 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:53.845814    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4rjg_3148d0bb-8388-4795-b985-e653b00e2b59/node-exporter/0.log"
Sep 18 21:10:53.874525 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:53.874493    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4rjg_3148d0bb-8388-4795-b985-e653b00e2b59/kube-rbac-proxy/0.log"
Sep 18 21:10:53.903976 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:53.903949    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4rjg_3148d0bb-8388-4795-b985-e653b00e2b59/init-textfile/0.log"
Sep 18 21:10:56.429396 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:56.429359    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/1.log"
Sep 18 21:10:56.439431 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:56.439401    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-x94rj_6f534f4e-ec58-41c6-a920-68a79947bdbc/console-operator/2.log"
Sep 18 21:10:57.128590 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128546    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"]
Sep 18 21:10:57.128906 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128888    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container"
Sep 18 21:10:57.128990 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128910    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container"
Sep 18 21:10:57.128990 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128934    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kube-rbac-proxy"
Sep 18 21:10:57.128990 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128943    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kube-rbac-proxy"
Sep 18 21:10:57.128990 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128952    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kube-rbac-proxy"
Sep 18 21:10:57.128990 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128962    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kube-rbac-proxy"
Sep 18 21:10:57.128990 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128972    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kube-rbac-proxy"
Sep 18 21:10:57.128990 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128980    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kube-rbac-proxy"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.128996    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129004    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129018    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129027    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129097    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kube-rbac-proxy"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129112    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kserve-container"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129121    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e6eb230-f0b0-4942-9b74-e7f13d116ebf" containerName="kube-rbac-proxy"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129130    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kserve-container"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129138    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="52585ff4-b97a-40ac-a56c-fc2ff98ce90f" containerName="kserve-container"
Sep 18 21:10:57.129368 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.129146    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f410f50a-2558-4e28-9da8-1285e20b4c9f" containerName="kube-rbac-proxy"
Sep 18 21:10:57.132222 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.132189    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.134773 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.134746    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wnf42\"/\"kube-root-ca.crt\""
Sep 18 21:10:57.134773 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.134759    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wnf42\"/\"default-dockercfg-6nnkt\""
Sep 18 21:10:57.135438 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.135423    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wnf42\"/\"openshift-service-ca.crt\""
Sep 18 21:10:57.142763 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.142744    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"]
Sep 18 21:10:57.237241 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.237195    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-proc\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.237241 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.237253    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-podres\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.237452 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.237275    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-sys\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.237452 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.237343    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-lib-modules\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.237452 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.237378    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvgtq\" (UniqueName: \"kubernetes.io/projected/08591219-07d8-4416-a0f8-e5058497982d-kube-api-access-wvgtq\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.338653 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-sys\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.338801 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-lib-modules\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.338801 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvgtq\" (UniqueName: \"kubernetes.io/projected/08591219-07d8-4416-a0f8-e5058497982d-kube-api-access-wvgtq\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.338801 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-proc\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.338801 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-podres\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.338801 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338746    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-sys\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.339023 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338852    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-lib-modules\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.339023 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338885    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-proc\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.339023 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.338962    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08591219-07d8-4416-a0f8-e5058497982d-podres\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.347655 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.347630    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvgtq\" (UniqueName: \"kubernetes.io/projected/08591219-07d8-4416-a0f8-e5058497982d-kube-api-access-wvgtq\") pod \"perf-node-gather-daemonset-qctfd\" (UID: \"08591219-07d8-4416-a0f8-e5058497982d\") " pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.440949 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.440924    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:57.717047 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.717023    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"]
Sep 18 21:10:57.719562 ip-10-0-137-48 kubenswrapper[2565]: W0918 21:10:57.719531    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod08591219_07d8_4416_a0f8_e5058497982d.slice/crio-bd28edb817331ee93b49e1656c0a7151f2b6137ee4faf81f31217f0849e6109d WatchSource:0}: Error finding container bd28edb817331ee93b49e1656c0a7151f2b6137ee4faf81f31217f0849e6109d: Status 404 returned error can't find the container with id bd28edb817331ee93b49e1656c0a7151f2b6137ee4faf81f31217f0849e6109d
Sep 18 21:10:57.782886 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:57.782859    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd" event={"ID":"08591219-07d8-4416-a0f8-e5058497982d","Type":"ContainerStarted","Data":"bd28edb817331ee93b49e1656c0a7151f2b6137ee4faf81f31217f0849e6109d"}
Sep 18 21:10:58.448497 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:58.448472    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6gc77_34fb603c-9f1a-46be-9e79-6f6a4f48527b/dns/0.log"
Sep 18 21:10:58.473654 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:58.473631    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6gc77_34fb603c-9f1a-46be-9e79-6f6a4f48527b/kube-rbac-proxy/0.log"
Sep 18 21:10:58.626289 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:58.626258    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkq9j_6fce8f58-7f3c-402c-bd7f-0318ccf013d3/dns-node-resolver/0.log"
Sep 18 21:10:58.789138 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:58.789051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd" event={"ID":"08591219-07d8-4416-a0f8-e5058497982d","Type":"ContainerStarted","Data":"17823e442c64d325b0f6dc3488d88fea5e1fb16a31862a1507de3ae5178a3c88"}
Sep 18 21:10:58.789138 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:58.789112    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:10:58.806576 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:58.806519    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd" podStartSLOduration=1.806502469 podStartE2EDuration="1.806502469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:10:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:10:58.806149054 +0000 UTC m=+3038.681079142" watchObservedRunningTime="2026-09-18 21:10:58.806502469 +0000 UTC m=+3038.681432534"
Sep 18 21:10:59.218537 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:10:59.218505    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8fkmr_f23a7662-58e8-41a3-80e3-bcf87e30af10/node-ca/0.log"
Sep 18 21:11:00.455898 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:00.455869    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c8cpg_d42f20c5-56ec-43f3-9d04-588f1608092e/serve-healthcheck-canary/0.log"
Sep 18 21:11:00.955756 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:00.955719    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pd9m9_a45d5039-6364-4256-bb67-b8f2fd3b336e/kube-rbac-proxy/0.log"
Sep 18 21:11:00.979137 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:00.979115    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pd9m9_a45d5039-6364-4256-bb67-b8f2fd3b336e/exporter/0.log"
Sep 18 21:11:01.002482 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:01.002458    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pd9m9_a45d5039-6364-4256-bb67-b8f2fd3b336e/extractor/0.log"
Sep 18 21:11:03.508194 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:03.508164    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-rv5wv_24ea8626-3cab-408b-a734-53697f31c346/manager/0.log"
Sep 18 21:11:03.555123 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:03.555094    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-l64jg_af1be1a5-f375-48bb-88ea-284cdf494981/server/0.log"
Sep 18 21:11:03.704671 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:03.704638    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-lsbkj_65afc7a3-249e-46d5-bf36-eb9adb0e7510/manager/0.log"
Sep 18 21:11:03.821019 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:03.820931    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-xgxz5_624f42b8-68d4-4789-b8b6-b4d7d4f89613/seaweedfs/0.log"
Sep 18 21:11:04.803860 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:04.803833    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wnf42/perf-node-gather-daemonset-qctfd"
Sep 18 21:11:10.185697 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.185671    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xznrr_76292b1c-a487-4039-8548-1cadba4c8fe4/kube-multus-additional-cni-plugins/0.log"
Sep 18 21:11:10.215713 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.215686    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xznrr_76292b1c-a487-4039-8548-1cadba4c8fe4/egress-router-binary-copy/0.log"
Sep 18 21:11:10.244899 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.244872    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xznrr_76292b1c-a487-4039-8548-1cadba4c8fe4/cni-plugins/0.log"
Sep 18 21:11:10.271196 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.271170    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xznrr_76292b1c-a487-4039-8548-1cadba4c8fe4/bond-cni-plugin/0.log"
Sep 18 21:11:10.306046 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.306018    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xznrr_76292b1c-a487-4039-8548-1cadba4c8fe4/routeoverride-cni/0.log"
Sep 18 21:11:10.356998 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.356968    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xznrr_76292b1c-a487-4039-8548-1cadba4c8fe4/whereabouts-cni-bincopy/0.log"
Sep 18 21:11:10.418089 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.418061    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xznrr_76292b1c-a487-4039-8548-1cadba4c8fe4/whereabouts-cni/0.log"
Sep 18 21:11:10.552864 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.552766    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ljrz9_803a9562-345e-4a66-9a13-095c205d61ee/kube-multus/0.log"
Sep 18 21:11:10.630453 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.630426    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l56h5_1c08e560-dab1-4bf9-b1a4-beec6c664021/network-metrics-daemon/0.log"
Sep 18 21:11:10.656254 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:10.656226    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l56h5_1c08e560-dab1-4bf9-b1a4-beec6c664021/kube-rbac-proxy/0.log"
Sep 18 21:11:11.944615 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:11.944579    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/ovn-controller/0.log"
Sep 18 21:11:11.966933 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:11.966908    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/ovn-acl-logging/0.log"
Sep 18 21:11:11.996325 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:11.996303    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/kube-rbac-proxy-node/0.log"
Sep 18 21:11:12.022202 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:12.022177    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/kube-rbac-proxy-ovn-metrics/0.log"
Sep 18 21:11:12.043012 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:12.042975    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/northd/0.log"
Sep 18 21:11:12.069600 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:12.069580    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/nbdb/0.log"
Sep 18 21:11:12.096803 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:12.096779    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/sbdb/0.log"
Sep 18 21:11:12.155817 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:12.155789    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vgfrj_85eb79b8-f47b-40da-a41c-94b053ddc4a3/ovnkube-controller/0.log"
Sep 18 21:11:13.189815 ip-10-0-137-48 kubenswrapper[2565]: I0918 21:11:13.189695    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-9rqxn_34b1b8ba-0fdc-433a-afab-64580eb8c2b2/network-check-target-container/0.log"